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PREFACE 


For  the  sixteenth  year,  the  Research  and  Theory  Division  of  the 
Association  for  Educational  Communications  and  Technology  (AECT)  is 
sponsoring  the  publications  of  these  Proceedings.  Papers  published  in 
this  volume  were  presented  at  the  national  AECT  Convention  *  in 
Nashville,  TN.  A  limited  quantity  of  this  volume  were  printed  and  sold. 
It  is  also  available  on  microfiche  through  the  Educational  Resources 
Information     Clearinghouse     (ERIC)  system. 

REFEREEING       PROCESS:  All    research    papers    selected  for 

presentation  at  the  AECT  convention  and  included  in  this  Proceedings 
were  subjected  to  a  rigorous  blind  reviewing  process.  All  references  to 
author  were  removed  from  proposals  before  they  were  submitted  to 
referees  for  review.  Approximately  fifty  percent  of  the  manuscripts 
submitted  for  consideration  were  selected  for  presentation  at  the 
Convention  and  for  Publication  in  these  Proceed  ings.  The  papers 
contained  in  this  document  represent  some  of  the  most  current  thinking 
in     educational     communications     and  technology. 

This   volume   contains   two   indexes.      The  first   is   an   author   index;  the 
second    is    a    descriptor    index.       The    index    for    volumes    1-6    (1979-84)  is 
included    in   the    1986    Proceedings,   and   the   index   for   volumes   7-10   Is  in 
the   1988  Proceedings. 

ML   R.  Simonson 
Editor 


-REVIEWERS- 
Research  Papers 


James  D.  Klein  -  Program  Planning  Chair 
Arizona  State  University 

Doris  R.  Pridcmore  -  Assistant  Program  Planner 
Arizona  State  University 

Gary  J.  Anglin 
Lexington,  Kentucky 

Anne  K.  Bednar 

Eastern  Michigan  University 

John  C.  Bel  land 
Ohio  State  University 

Roy  M.  Bohlin 
California  State  University 

Janet  L.  Bohren 
University  of  Cincinnati 

Dean  L.  Chris  tense  n 

Control  Data  Corporation,  Minnesota 

Gayle  V.  Davidson 
University  of  South  Alabama 

Ann  DcVancy 
University  of  Wisconsin 

John  V.  Dcmpsey 
University  of  South  Alabama 

Marcy  Driscoll 
Arizona  State  University 

Thomas  M.  Duff/ 
Indiana  University 

Norman  C.  Hi  gins 
Arizona  State  University 

Simon  Hooper 
University  of  Minnesota 

David  It.  Joit assert 

University  of  Colorado  at  Denver 

Mahle  II.  Klnxlc 
lliailotlcsvlllt\  Virginia 

Nancy  Nelson  Knttpfi 
KiittKUft  Slate  Univcrs 

Raymond  Ktilhuvy 
Arizona  State  Univcn 


Kathryn  L.  Ley 

G rambling  State  University 

Gary  R.  Morrison 
Memphis  State  University 

Jacqueline  K.  ODell 
University  of  Arkansas 

Tillman  J.  Ragan 
University  of  Oklahoma 

Lloyd  Reiber 

Texas  A&M  University 

Landra  L.  Rezabek 
Florida  State  University 

Rhonda  S.  Robinson 
Northern  Illinois  University 

Steven  M.  Ross 
Memphis  State  University 

Wilhelmina  C.  Savenye 
Arizona  State  University 

Michael  R.  Simonson 
Iowa  State  University 

Patricia  L.  Smith 
University  of  Oklahoma 

Michael  J.  Stteibel 
University  of  Wisconsin 

Howard  J.  Sullivan 
Arizona  State  University 

Susan  A,  Tucker 
University  of  South  Alabama 

John  F.  Wedman 
University  of  Missouri 

Brent  G.  Wilson 

University  of  Colorado  at  Denver 

William  Winn 
University  of  Washington 


5 


RESEARCH  AND  THEORY  DIVISION  OFFICERS 


President: 


Brent  G.  Wilson 

University  of  Colorado  at  Denver 

Campus  Box  106 

P.O.  Box  173364 

Denver,  CO  80217-3364 

Day  303-556-4363 

Fax  303-556-4479 

e-mail  bwilson@cudnvr.denver.colorado.edu 


President-elect: 


Gary  R.  Morrison 
Memphis  State  University 
972  Grouse  Meadow  Dr. 
Cordoba  TN  38018 
901-678-3414 

e-mail  morrison.gary@coe.memst.edu 


Past  President: 


James  D.  Klein 

Arizona  State  University 

College  of  Education/FPE  -  061 1 

Tempe,  AZ  85287-0611 

602-965-3384 

Fax  602-965-0300 

e-mail  atjdk@asuacad  (bitnet) 


Publications  editor: 


Kathryn  L.  Ley 
Grambling  State  University 
14  University  Blvd 
Ruston,  LA  71270 
318-274-2364 
Fax  318-274-2656 


Board  Members: 


Gayle  V.  Davidson 

University  of  Texas 

Dept.  of  Curriculum  &  Instruction 

Austin,  TX  78712 

512-471-5211 


Simon  Hooper 
3212  Croft  Drive 
St.  Anthony,  MN  55418 
612-625-0534 


Marcy  P.  Driscoll 
Florida  State  University 
Educational  Research  B 1 97 
Tallahassee,  FL  32306 
904-644-4592 

Barbara  L.  Grabowski 
Penn  State  University 
128  W.  Prospect  Avenue 
State  College,  PA  16801 
315-443-3704 


Landra  L.  Rezabek 
Florida  State  University 
College  of  Education 
307  Stone  Building,  B-197 
Tallahassee,  FL  32306-3030 

Willi  Savenye 
Arizona  State  University 
Div.  of  Psychology  in  Educ. 
Tempe,  AZ  85281 


9 

ERIC 


e 


PREVIOUS  PROCEEDINGS 
ERIC  ED  NUMBERS 


YEAR 

LOCATION 

ED  NUMBER 

1979 

New  Orleans 

171329 

1980 

Denver 

194061 

1981 

Philadephia 

207487 

1982 

Dallas 

223191  to 
223236 

1983 

New  Orleans 

231337 

1984 

Dallas 

243411 

1985 

Anaheim 

1986 

Las  Vegas 

267753 

1987 

Atlanta 

285518 

1988 

New  Orleans 

295621 

1989 

Dallas 

308805 

1990 

Anaheim 

323912 

1991 

Orlando 

334969 

1992 

Washington,  DC 

347970  to 
348041 

< 

9 

ERIC   


Association  for  Educational  Communications 
and  Technology 

What  Is  AECT? 

AFCT  is  the  only  national,  professional  association  dedicated 
to  the  improvement  of  Instruction  through  the  effective  use  or 
media  and  technology.  AECT  assists  Its  members  In  using 
technology  In  their  jobs  and  to  enhance  the  learning  process. 


Who  belongs  to  AECT? 

U  Media  Specialists 

U  Educators 

U  Librarians 

□  instructional  designers 

□  Corporate/mWtary  trainers 

U  Learning  resource  specialists 

U  Curriculum  developers 

U  Television  producers  and  directors 

U  Communications  specialists 

U  Education  administrators 

U  Olheis  who  require  expertise  in  Instructional  technology 


What  are  AECT  members  Involved  In? 

U  Hypermedia 
U  interactive  Video 
U  CD-ROM 

u  ca 

U  Teleconferencing 

U  Film  A  video  utilization 

U  Telecommunications 

U  Computer  software  ft  hardware 

U  Projection/presentation  products 

U  Intelligent  tutoring  systems 

U  Videodiscs 

U  Distance  learning 

U  And  morel 

AECTs  publications,  convention*  trade  show,  and 
conferences  present  the  leading  edge  on  research  and 
practical  applications  for  these  and  other  technologies. 

AECT  History 

Unlike  some  other  special-Interest  technology  organizations, 
AECT  has  a  long  history,  with  over  70  years  in  educational 
technology.  We've  grown  up  with  technology,  advocating  Its 
Integration  Into  education  from  «ms  to  Integration  Into 
education  from  films  ta  overheads  to  Interactive  video  and 
hypermedia. 


AECT  began  os  the  Department  of  Visual  Instruction  at  the 
National  Education  Association  in  1923.  in  the  days  when  visual 
olds  consisted  of  «ms  and  Hides.  In  1947.  as  educators  were 
adapting  technology  used  to  train  World  War  I  service 
personnel  for  the  classroom,  the  name  of  the  organization 
became  the  Deportment  of  Visual  Instruction  (DAV1).  Twelve 
years  later.  DA VI  became  an  afMate  of  the  NEA  and  finely  the 
autonomous  association.  AECT.  in  1974. 

Today.  AECT  keeps  on  eye  on  the  future  of  instructional 
technology  whfe  assisting  educators  with  the  changes  and 
chalenges  that  face  them  now.  AECT  members,  now 
numbering  4£00  are  professionals  devoted  to  quality 
education. 

They  care  about  doing  their  Jobs  better  and  wont  to  embrace 
new  methods,  new  equipment,  and  new  techniques  that  assist 
learning. 

Membership  In  AECT  Increases  your  effectiveness,  your 
expertise,  and  your  skMs.  These  qualities  In  turn  enhance  your 
professional  Image  and  earning  potential. 


AECT  Affiliates 

AECT  has  47  state  and  16  regional  and  national  a  filiates,  and 
has  recently  established  several  chapters  surrounding  major 
universities  and  metropolitan  areas.  These  affiliated 
organizations  add  a  localized  dimension  to  your  AECT 
membe.  J^p  and  aHow  for  more  Interaction  among  your 
colleagues.  For  more  detain  on  chapters  and  affMates  in  your 
area,  contact  the  AECT  National  Office. 


AECT  National  Convention  and  InClTE 
Exposition 

Each  year.  AECT  brings  top  speakers  to  exciting  locations,  and 
presents  over  300  sessions  and  special  events  to  provide  the 
best  training  available  In  the  use  of  media  in  education  and 
Instruction.  The  convention  features  the  hCiTE  Exposition,  the 
first  trade  show  created  exclusively  for  Instructional 
technology  products.  At  InClTE.  youl  see  computers,  learning 
systems,  software.  Interactive  multimedia,  audiovisual 
products,  flms  ft  videotapes,  projectors  and  presentation 
products,  video  eqiipment.  accessories,  and  more.  The 
Convention  offers  tracks  of  sessions  focusing  on  specific 
Interest  areas  surrounding  AECTs  nine  Divisions  and  other 
special  interests.  The  convention  has  featured  a  Hypermedia 
Strand  and  a  Total  Quality  Management  Track.  In  addition. 
Intensive  ful  ond  half-day  workshops  are  offered  tor  in-depth 
training  on  the  latest  technology  applications  for  education. 


Research  and  Theory  Division  (RTD)  improves  the 
design,  execution.  utMzation.  evaluation,  and  dissemination  of 
educational  technology  research  and  theory;  advtses 
educators  on  using  research  results. 
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□□  Yourself  a  Favor . 


JOIN  AECT  TODAY! 

AECT  ti<ij  mne  special  interest  divisions  (o  inccl  your  needs  and  address  your  special  interests  within  the  field  ol  educational  technology  and  media  Each  division 
publishes  its  own  newsletter  and  is  devoled  lo  research,  communications  and  pubficalion  ol  findings  within  its  specialty  In  addition,  most  divisions  sponsor  an 
awards  program  Your  AECT  membership  includes  one  division,  and  you  may  join  as  many  divisions  as  you  like  lor  $10  each 


Media  Oeslfn  and  Production  Division  (MDPD)  locuses  on  enhancing  media 
tn'Hluclion  'Jtills  and  provides  a  towm  lor  interaction  among  researchers, 
cwnnrtcia!  and  noticominctrial  media  producers.  -.J  educalors 

Imfutfriaf  Trafnin§  and  Edocatton  Division  (ITtO)  is  involved  wilti  designing. 
liMiKitng  <md  managing  training  programs,  and  works  to  promote  ma»in<im 
utilization  of  educalional  techniques  and  media  that  are  ellerJive  m  practical  use 

Division  of  Interactive  Systems  and  Compvters  (DISC)  is  concerned  with 
tie  design  devnloprnerrf  cfchvery.  evaluation  and  tnfegratinn  ot  interactive  media 
and  computer  based  applications  in  education  and  training  The  Division  promotes 
(he  networking  ot  its  members  lo  tanilirate  sharinp  ot  expertise  and  interests 

Division  of  Instructional  Development  (DID)  studies,  evaluates,  and  refines 
drctqn  processes,  cieales  new  models  of  instructional  envelopment.  rJisseminates 
Imdiitqv  and  oromntes  academic  programs 

International  Oivisioo  (tNU)  rucourages  practice  and  research  in  educational 
coiTiniimotiOn  and  distance  education  tor  social  and  economic  development  across 
national  and  ctittwal  Imes  It  promotps  international  exchange  and  sharing  ot 
mfnmaliofi.  and  enhances  professional  and  personal  relationships  between  present 
and  future  international  leaders 


Division  of  EiocoNoal  Media  MaMfomtwt  (DIMM)  creates  gu«W«es  on 
educational  media  management  and  sponsors  programs  to  enhance  media 
management  skiNs  The  Oivtston  strives  to  resolve  common  problems  •rttwi  me 
industiy  and  design  practical  solutions 
Hit  earth  and  Theory  DfcritSvw  (RID)  lacdiWes  the  design,  eiecutmn. 
utilization,  evaluation,  and  dissemination  ol  educational  technology  research, 
promotes  applied  and  theoretical  research  on  the  use  ot  educational  technology 
and  encourages  me  use  of  rnuftple  restart  paradigms  m  curnmg  issues  related 
to  technology  in  instruction 

OhrfslM  ol  School  Medio  Specialists  (DIMS)  promotes  wmmurocations 
among  school  media  personnel  who  share  a  common  concern  in  the  development 
incrementation,  and  evaluation  ol  school  media  programs;  and  sinves  lo  increase 
learning  and  improve  instruction  in  Vie  school  setting  tirough  Vie  Urination  ol 
educational  media  and  technology. 

DhrlslM  at  Telecofwwawlcatiows  (DOT)  improves  instruction  through  the  use 
of  lelecormwkations  including  television,  radio,  video,  and  audio  devices  The 
Division  is  concert  with  design,  production,  utilization,  and  evaluation  ot 
instructional  teleconvnunir^iorrs  materials  and  equipment,  as  well  as  me  training 
and  Qualifications  of  personnel 


YES! 


 MEMBERSHIP  ENROLLMENT  FORM  

liiiotl  me  tor  the  membership  Mtcgmy.  divisions),  and  options  I  have  selected  below  I  understand  that  my  membership  will  pxtend  tor  12 
months  Irurn  lite  date  my  payment  is  received 


I  t  Regular  Membership.  $65  fee  includes  $12  for  leUiltenrh  and  one  tree 
division 

I  I  Intemalionaf  Regular  Membt rshlp.  $77  (surlace  mail) 

I  t  International  Rtgufar  Membership.  $107  (air  mad) 

I  l  Comprehensive  Membership.  $95  Tee  includes  $1?  lor  lethhends.  $30 

tor  idrn'Mtonat  Technology  Research  &  Development,  and  one  Irre  division 
I  t  International  Comprehensive  Membership.  $115  (surface  mad) 
I  I  IntematfonaT  Crnnprehenshre  Membership.  $14$  (air  mad) 
I  I  Student  Membership.  Itf*  tee  includes  $1?  tor  lechhcnds  km)  one  tree 

division 

I  I  International  Student  Membership.  $36*  (surtace  nvnl) 
I  I  Comprehensive  Student  Membership.  $4S*  fee  includes  $12  for 
lechlrends.  $20  lor  educational  Technology  Research  £  Development,  and  one 

line  division 

I  I  International  Comprehensive  Student  Membership.  $66*  (surtace  mart) 
l  t  Corporate  Membership,  $310  fee  includes  $12  tor  TechTrenfc  and  one 
tier"  riwr.ion 


'  Ntih'  Minimi  a^dii-anK  iihi:J  or  rjHotH  m  lotinal  coursrwork  in  Ihe 
fdnr.iiimul  hitKvilorrv  ItHd  rm  at  teasl  a  hall  lime  basis  to  quality  tor  this 
s|sn  ul  i.M«*  Appliuuts  iihisI  im  liKfc  Hie  name  of  11*0  uniwisity  and  head  ot 
lite  ifcittflflinil  lor  vrtthciliuii 

Kwv  id  thuvtisilv  -  


DdMrl'ittnt  Head  


.Phone. 


OhrlslM  Memberships  Please  choose  your  one  tree  division  Irom  hsl  above 

(IDI5C    HOOT     GRID     DOIO  QINTL 

tllTfO     LJDCMM  DMOPO  DDSMS  %  TREE 


1 1DISC 
tllTEO 


($10  each  additionat  Divisional  affiliation) 
flOOT     ClftTD     DOID  OINTl 
DDCMM  DMOPO  DOSMS    Total  extra  divisions $. 

Tout  Memberships. 


t  ?My  check  or  money  order  is  enclosed. 
(  I  Charge  tms  order  to  my  credii  card. 

D  Mastercard  DVisa 


Card  *- 
Signature  _ 


.Expires, 


(1  Please  biH  me 
Man*.  Title  


Institution  . 


.Phone- 


Address. 


City.  Stale.  Zip. 


Mall  Me  term  la  AECT,  It?)  Vermont  Ave.,  me,  WatbJaejbM,  DC  2t$t$. 
or  rai  ynr  Ui  rm  n  peyloa  by  or**  tare. 


f  *«s  to  fir  Association  tot  idiMltonal  Communications  and  Irchnoloqy  may  he  deductible  by  members  for  federal  income  la*  purposes  as  ordinary  and  necessary 
busmen  expenses  Dues  and  gifts  are  not  deductible  chautaNe  contributions  Pkise  consult  your  tax  adviser  lor  assistance  in  your  specific  situation 


sect 
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ABSTRACT 


While  innovation  advocates  have  focused  on  the  need  to  reform  Philippine 
Hvhtndn  to  meet  the  needs  and  demands  of  society,  little  has  been  done  to  change 
thv  teachers1  practices,  Recent  studies  have  pointed  to  inadequate  teacher 
training  as  one  of  thv  reasons  for  Filipino  students'  low  achievement.  This  study 
was  conducted  to  identify  specific  variables  which  contributed  significantly  to 
teacher  competence,  affect,  familiarity  and  perceived  skill.  Four  independent 
variables  were  analyzed:  place  of  work,  academic  subject,  years  of  teaching 
experience  and  training  in  instructional  technology.  Findings  suggest  specific 
variables  significant  in  designing  a  curriculum  for  preparing  teachers  in 
implementing  educational  programs. 
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Purpose  of  the  Study 


This  study  was  undertaken  in  response  to  recent,  widespread  calls  for  reforms  and 
improvements  in  Philippine  s<  nools,  secondary  schools  in  particular  (Congressional 
Commission  on  Education,  V.  91;  Cortes,  1990;  Quisumbing,  1987;  Sutaria,  1989).  It  has 
been  acknowledged  that  changing  these  schools  will  require  changing  teachers*  practices, 
since  teachers  have  ft.*  major  responsibility  for  implementing  educational  programs 
(Gonzales,  1986).  Innovation  advocates  recognize  the  contribution  of  technology  in  reforming 
the  school  system.  The  technology  brought  earlier  to  the  military  (Hush,  1990)  and  industry 
(Geber,  1990)  has  set  the  pace  and  opened  great  promises  of  what  technology  ran  bring  to 
education.  Instructional  technology  (IT)  with  its  system  of  designing,  implementing  and 
evaluating  instruction,  seems  to  offer  a  body  of  principles  and  vtuhniques  that  can  improve 
instructional  practice  and  curriculum  dev  iiopment.  This  study  was  undertaken  to 
determine: 

1 .  How  competent  are  Philippine  teachers  in  the  use  of  instructional  design  and 
technology? 

2.  7o  what  extent  do  Philippine  teachers  value  instructional  design  and 
technology? 

3.  How  familiar  are  Philippine  teachers  with  instructional  design  and  technology? 

4.  To  what  extent  do  Philippine  teachers  believe  that  they  are  skilled  in  the  use  of 
instructional  design  and  technology? 


Literature  Review 

Studies  of  Philippine  education  from  the  Monroe  Sur\ey  in  1925  (Cortes,  1993)  to  the 
Congressional  Commission  on  Education  Report  in  1991  have  consistently  reported  the  low 
levels  of  achievement  among  students.  The  findings  showed  that,  "Pupils  on  the  average 
learn  only  55  percent  or  even  less  of  what  must  be  learned  at  every  grade  lever 
(Congressional  Commission  on  Education  Report,  1991,  p.  10).  The  Congressional 
Commission  on  Education  Report  has  generated  a  lot  of  concern  from  both  the  government 
and  non-government  sectors.  Rama  (1993)  expounds  the  theory  of  economic  development 
being  dependent  on  the  literacy  level  of  the  people,  noting  that  The  better  educated  the 
people,  the  more  prosperous  a  country.  By  the  same  token,  the  poorer  the  standard  of 
education,  the  poorer  the  country"  (p.  4).  Unfortunately  while  the  1990  Philippine  Census  of 
Population  and  Housing  claims  93.5  percent  literacy  (cited  in  Braid,  1993),  the  Functional 
Literacy  Education  and  Mass  Media  Survey  establishes  only  73.2  percent  of  those  ten  years 
old  and  above  as  functionally  literate  (cited  in  Santos,  1993).  Braid  (1993)  def/nes 
functional  literacy  as  enabling  "the  individual  to  survive  and  function  within  his  society"  (p. 
16). 

Education  which  can  help  propel  the  Philippines  to  its  dream  of  economic  prosperity 
in  the  year  2000  (Braid,  1993)  requires  quality  teachers.  "Time  and  again  the  success  of 
educational  reforms  has  been  found  to  be  decisively  dependent  on  ...  the  quality  of  teachers. 
No  education  system  can  rise  too  far  beyond  the  level  of  the  teachers"  (Roy  Singh,  1991,  p. 
75).  If  teachers  are  not  proficient  to  teach,  the  quality  of  education  equally  suffers.  Cortes 
(1993)  attributes  the  problem  of  quality  education  to  varied  factors,  but  stresses  that  the 


key  to  quality  education  are  the  teachers.  An  earlier  study  (Cortes,  1987)  has  confirmed 
that  the  teacher  factor  contributes  to  the  low  level  achievement  of  Filipino  students.  Equally 
the  Congrosrtional  Commission  on  Education  Report  (1991)  singled  out  the  teacher  as  the 
most  important  factor  in  education  whose  influence  as  a  change  agent  in  society  is  beyond 
question. 

Earlier  reports  on  teacher  competence  have  shown  the  low  qualifications  of  many 
teachers  to  teach  subjects  they  have  been  assigned  to  teach  (Gonzales,  1989;  Sutaria,  1989; 
Tarvin  &  Faraj,  1990).  This  condition  could  be  traced  to  poor  teacher  training 
(Congressional  Commission  on  Education  Report,  1991).  To  correct  the  flaws,  the 
Congressional  Commission  on  Education  Report  (1991)  has  recommended  two  important 
points  among  others:  1)  strengthening  pre-service  teacher  education,  and  2)  improving  and 
expanding  the  in-service  training  programs  for  teachers. 

Quality  teaching  is  defied  operationally  as  skills  in  teaching,  production  of 
instructional  media  and  test  and  measurement  (Salinas,  1989).  Teachers  are  expected  to 
have  solid  background  in  the  psychology  of  teaching  and  learning,  a  variety  of  teaching 
strategies  and  an  adequate  knowledge  of  what  to  teach  (Cortes,  1993).  Philippine 
education  calls  for  reforms  which  include  a  system  to  improve  instructional  practices,  The 
field  of  instructional  technology,  with  a  system  in  designing,  implementing  and  evaluating 
the  total  process  of  instruction  (Reiser,  1987)  offers  the  mode  of  reform  for  Philippine 
education. 

Although  before  this  study  reported  herein  there  had  been  no  studies  involving 
Philippine  secondary  school  teachers'  competence,  affect,  familiarity  and  perceived  skills 
regarding  instructional  design  and  technology,  generalizations  regarding  specific  variables 
treated  in  this  study  have  appeared  in  some  literature. 

Knupfer's  study  (1988)  showed  that  teachers  in  urban  areas  had  more  in-service 
training  than  teachers  in  rural  areas.  Training  was  observed  to  have  a  significant 
correlation  to  positive  attitude  towards  innovations  (Burke,  1986;  Knupfer,  1988;  Mims, 
1975;  Salinas,  1989;  Streeter,  1967;  Woolsey,  1986).  In-service  training  consistently 
improved  teachers'  performance  (Cortes,  1993). 

Dawson  (1977)  found  physical  science  majors  to  have  significantly  higher  scores  on 
the  IT  Attitude  Scale  than  social  sciences  majors.  Likewise  physical  science  teachers  were 
noted  to  have  more  positive  attitude  towards  microcomputers  than  the  social  sciences 
teachers  (Fary,  1989). 

Ewing's  study  (1986)  reveak...  «nai  the  use  of  IT  was  a  function  of  users'  experience 
and  familiarity  with  technology.  Computer  anxiety  was  affected  by  lack  of  skills  and 
resistance  declined  with  familiarity  and  experience  with  automated  technology  (Albritton  & 
Sievert,  1984).  Van  Duinen  (1975)  noted  in  his  study  that  sex,  educational  attainment,  and 
number  of  years  in  the  teaching  service  significantly  influence  the  expressed  knowledge  of, 
attitude  towards  and  experiences  of  teachers  in  instructional  development. 

It  may  be  concluded  from  this  summary  of  literature  review  that  to  introduce  change 
in  schools  a  concerted  effort  should  be  focused  on  the  teacher,  the  key  implementor  of  change. 
Probably  among  the  important  variables  to  influence  teachers'  competence,  affect,  familiarity 
and  perceived  skills  regarding  instructional  design  and  technology  are  teachem'  plan1  of 
work,  field  of  specialization,  training,  and  years  of  teaching  experience.  All  thm*  criteria 
were  included  in  this  study  in  identifying  training  needs  of  Philippine*  Herutuiary  hciIhmi! 
toucher*. 
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Method 


The  survey  research  method  employing  face-to-face  data  collection  was  used  in  this  study. 


Subjects 

Subjects  were  389  Philippine  secondary  school  teachers  drawn  via  stratified,  multi- 
stage, random  sample  to  represent  rural  (Albay),  urban  (Iloilo),  and  metropolitan  (Manila) 
areas.  These  teachers  represented  both  physical  science  and  social  science  instructors 
(academic  subject),  with  teaching  experience  of  from  1  to  20+  years,  and  with  different  levels 
of  training  (College  plus  Inservice,  College,  Inservice  and  No  Training)  in  instructional 
technology.  These  four  factors  were  the  independent  variables  in  the  study.  Complete  data 
were  obtained  on  370  (95%)  of  the  sample. 


Instruments 

A  three-part  instrument  was  developed  for  the  study.  Part  I  -  Competence  Test  - 
was  a  test  of  instructional  design  skills.  This  portion  of  the  instrument  (21  items)  focused 
on  the  seven  instructional  design  components  described  in  Dick  and  Reiser's  (1989)  book, 
Planning  Effective  Instruction.  Part  II  -  Instructional  Technology  Scale  (IT  Scale)  -  of  the 
instrument  consisted  of  38  phrases  with  an  accompanying  Likert  scale  to  assess  the 
respondents'  perceived  skill,  familiarity,  and  values  regarding  instructional  media  and 
instructional  design.  Part  III  of  the  instrument  dwelt  on  demographic  information  such  as 
place  of  work,  field  of  specialization,  years  of  teaching  experience  and  training  in 
instructional  technology.  The  instrument  was  pilot  tested  before  it  was  used  in  the  study. 


Procedures 

In  most  schools  the  primary  investigator  was  able  to  gather  the  teachers  in  a  single 
location  to  administer  the  research  instrument.  The  exception  was  in  Manila  where 
overcrowding  in  classrooms  and  the  consequences  of  recent  natural  disasters  in  the 
Philippines  required  that  school  principals  supervise  the  distribution  and  collection  of  thr, 
completed  instrument.  Completed  questionnaires  were  picked  up  by  the  primary 
investigator  on  the  nvernge  of  one  week  from  the  date  of  delivery.  The  data  collection  took 
about  two  months  as  it  required  travel  both  by  land  and  air  to  gather  the  materials  from 
the  different  island*  in  the  country. 


Datu  Analysis 

Data  gathered  from  Part  I  of  the  questionnaire  were  coded  according  to  correct 
answers,  while  demographic  data  (Part  III)  were  coded  according  to  each  respondent's 
region,  province/city,  years  of  teaching  experience,  educational  level  completed,  age,  academic 
subject,  and  IT  training,  Part  II  of  the  questionnaire  was  coded  in  the  same  way  as  the 
five-point  scale  used:  5  •  A  complete  extent;  4  •  A  considerable  extent; 
3  ■  A  moderate  extent;  2  -  A  very  little  extent;  and  1  •  No  extent.  The  completed  answer 
sheets  were  optically  scanned.  Besides  descriptive  statistics,  ANOVA  was  used  to  test  for 


significant  differences  in  competence,  perceived  skill,  familiarity,  and  values  based  upon  the 
four  independent  research  variables    region,  academic  subject,  training  in  instructional 
technology  and  years  of  teaching  experience.  Where  variables  yielded  a  significant  F  value,  a 
Scheffe  Post  Hoc  Test  was  used  to  detect  which  mean  differences  contributed  to  the 
significance.  Correlational  and  partial  correlational  analyses  were  used  to  determine  he 
relationships  among  the  study's  dependent  variables    competence,  perceived  skill, 
familiarity,  and  values  regarding  instructional  design  and  technology.  All  data  were 
analyzed  using  the  Statistical  Analysis  System. 

The  questionnaire  was  tested  for  internal  consistency  reliability.  The  resulting 
Cronbach  alpha  for  each  scale  of  the  questionnaire  was  determined.  The  reliability 
calculations  were  processed  using  the  ANLITH  system. 

For  the  purpose  of  reporting  the  data,  the  average  Likert  scale  ratings  were  divided 
by  5  and  results  reported  as  percentage.  Mean  rsw  scores  on  the  competence  test  were 
divided  by  45  and  reported  as  percentage. 


Competence  of  Philippine  teachers  in  the  use  of  instructional  design  and  technology 

The  data  pertinent  to  research  question  1  show  that  the  mean  proficiency  score  was 
60%-  The  Cronbacks  alpha  for  the  proficiency  test  was  .56. 

Significant  differences  were  observed  among  the  means  of  the  teachers  grouped 
by  levels  of  training  in  instructional  design  and  technology.  The  groups  which 
contributed  to  the  significant  F  value  were  the  College  plus  Inservice  (teachers  who  have 
College  and  Inservice  training  in  instructional  technology)  and  the  Inservice  training 
groups.  No  significant  differences  were  observed  among  the  mean  responses  of  teachers 
grouped  by  region,  academic  subject,  and  years  of  teaching  experience. 

Two  notable  findings  were  observed  when  means  of  perceived  skill,  familiarity, 
and  value  were  compared  with  the  proficiency  means  (see  Table  1).  Albay  consistently 
ranked  lowest  in  self-rated  perceived  skill,  familiarity,  and  value  regarding  instructional 
design  and  technology,  but  ranked  highest  on  the  competence  test. 


Results  and  Analysis 
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Table  1 

Means  of  Respondents  Grouped  by  the  Four  Independent  Research  Variables  on  the  Four 

Research  Scales  Converted  to  Percentage 


Group 

Familiarity 

Skill 

Value 

Competence 

Alb  ay 

HA  ft 
/  4.D 

ftft  ft 
OO.O 

7  A  ft 
/4.0 

ftft  ft 
OU.O 

HOIK) 

/O.o 

ftQ  ft 

OS7.0 

7R  ft 
/  D.O 

KQ  ft 

oy.o 

iviuniia. 

7ft  n 
/o.u 

7 1  n 

7fi  ft 
/  D.O 

oy.u 

/ 

rnysitdi  01. 

7fi  A 
/  0.4 

ftQ  ft 

7fi  n 
r  o.u 

ftft  7 

ou.  / 

OOC1H1  uC, 

7ft  n 

7ft  >1 
/U.4 

7K  A 
#0.4 

c;ft  o 
oo.y 

onegcTi  nservice 

7Q  ft 

/y.o 

7>1  /I 
l  4.4 

ftn  ft 
ou.o 

RA  ft 
04.0 

vxniege 

m\  ft 

/  o.U 

70  ft 

/y.u 

fift  vl 

OU.4 

Inservice 

/b.4 

/U.4 

7K  ft 
/D.O 

CO  4 

Do. 4 

Mr* -TS»m  hi  ti  or 

I^IU"  AACIUllIlgJ 

79  4. 

DO.  Ht 

79  ft 

fin  ft 
ou.o 

0-1  year 

78.8 

66.2 

80.8 

60.6 

2-3  years 

72.4 

64.8 

73.2 

62.0 

4-5  years 

68.4 

61.2 

65.8 

58.8 

6-7  years 

70.6 

83.2 

70.0 

60.3 

8-9  years 

71.0 

66.2 

71.8 

61.2 

10-11  years 

71.0 

62.0 

69.4 

58.7 

1213  years 

73.2 

64.8 

71.8 

59.2 

14-15  years 

69.6 

65.0 

70.8 

57.0 

16-17  years 

68.8 

61.4 

69.2 

59.4 

18- 19  years 

74.2 

66.4 

74.4 

62.4 

20  &  above  years 

69.6 

64.2 

69.2 

57.9 

Similarly,  an  unusual  finding  was  observed  among  the  No-Training  group  which 
perceived  themselves  to  be  less  familiar  with,  and  less  skilled  in  the  use  of  instructional 
design  and  technology.  They  also  ranked  lowest  in  value  toward  instructional  design  and 
technology.  The  mean  score  of  the  No-Training  group  on  the  competence  test,  although  not 
statistically  different  from  the  other  training  groups,  ranked  second  highest. 

Competence  and  perceived  skill,  familiarity,  and  value  showed  little  correlation,  - 
which  means  that  knowledge  of  the  perceived  skill, 

familiarity,  and  value  were  not  reliable  indicators  of  teachers'  actual  competence  in  the 
use  of  instructional  design  and  technology. 

As  a  predictor  of  competence,  perceived  skill  and  value  were  found  to  be 
significant;  however,  the  most  important  predictor  of  the  competence  score  was  training. 
Training  accounted  for  2.9%  (see  Table  2)  of  the  competence  score. 
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Table  2 


Summary  Table  Comparing  Familiarity,  Perceived  Skill,  Value,  and  Training  with 
Competence  Scores  (Partial  Correlation) 


Source  df 

SS 

MS  F 

P 

r2 

r 

Familiarity 

1 

48.02 

48.02 

2.29 

0.132 

.0058 

.076 

Skill 

1 

170.17 

170.17 

8.11 

0.004 

.0207 

.144 

Value 

1 

149.21 

149.21 

7.07 

0.008 

.0180 

.134 

Training 

3 

237.65 

79.21 

3.77 

0.011 

.0289 

.170 

Error 

363 

7610.02 

20.96 

Total 

369 

8215.07 

Values  of  Philippine  teachers  regarding  instructional  design  and  technology 

The  findings  pertinent  to  research  question  2  indicated  a  moderate  extent  of  value 
toward  selected  components  of  instructional  design  and  technology.  All  four  comparison 
groups  reported  a  mean  value  which  fell  within  the  range  of  moderate  extent  of  value. 

Significant  differences  in  the  mean  value  of  instructional  design  and  technology  were 
observed  among  teachers  grouped  according  to  level  of  instructional  design  and  technology 
training.  The  main  contributor  to  the  significant  F  value  was  the  mean  difference  between 
the  College  plus  Inservice  and  the  No-Training  groups. 

There  were  no  significant  differences  in  the  mean  value  regarding  instructional 
design  and  technology  among  the  three  regions,  the  two  academic  subjects  and  the  1 1 
groups  of  teachers  grouped  according  to  years  of  teaching  experience  selected  for  the  study, 

An  examination  of  the  mean  responses  by  the  four  independent  research  variables 
(see  Table  1)  showed  that  value  was  highest  among  beginning  teachers  (0-1  year)  and 
lowest  among  those  with  4-5  years  of  teaching  experience.  Overall,  means  of  value  ranked 
second  to  those  on  familiarity,  while  perceived  skill  ranked  lowest  among  the  self-reported 
variables. 

High  positive  correlations  were  observed  between  expressed  value  and  familiarity 
and  between  value  and  perceived  skill.  The  correlation  between  value  and  competence  was 
statistically  significant,  but  the  correlation  coefficient  of  r=.123  was  not  particularly 
important.  As  a  predictor,  value  accounted  for  1.8%  (see  Table  2)  of  the  competence  score. 

The  Cronhack's  alpha  for  the  Instructional  Technology  Scale  on  the  value  variable 
was  .94. 


Familiarity  of  Philippine  teachers  with  instructional  design  and  technology 

The  findings  pertinent  to  research  question  3  indicated  a  moderate  extent  of 
familiarity  with  selected  components  of  instructional  design  and  technology.  All  four 
comparisons  reported  a  mean  value  which  fell  within  the  range  of  moderate  extent  of 
familiarity.  The  medians  were  generally  higher  than  the  means,  which  indicated  that  there 
were  more  teachers  who  perceived  themselves  familiar  with  selected  components  of 
instructional  design  and  technology  than  those  who  perceived  themselves  as  less  familiar. 


Significant  differences  were  observed  among  teachers  grouped  by  level  of  training  in 
instructional  design  and  technology.  The  main  contributors  to  the  significant  F  value  were 
the  mean  differences  between  the  College  and  the  No-Training  groups  and  between  the 
College  plus  Inservice  and  the  No  Training  group. 

No  significant  differences  were  noted  on  the  mean  responses  of  teachers  from  the 
three  regions  studied,  teachers  from  the  Physical  Sciences  and  the  Social  Sciences,  or  the 
mean  responses  of  teachers  grouped  by  years  of  teaching  experience. 

An  examination  of  the  mean  responses  by  the  four  independent  research  variables 
(see  Table  1)  showed  that  familiarity  wiih  highest  among  beginning  teachers  and  lowest 
among  those  with  4-5  years  of  touching  experience.  Overall,  mean  responses  of  familiarity 
ranked  highest  followed  by  the  mean  responses  in  value  and  perceived  skill. 

High  positive  correlations  between  familiarity  and  perceived  skill  and  between 
familiarity  and  value  were  observed,  Tin?  correlation  between  familiarity  and  competence 
was  low  (r=.()(M).  Fmnilinrity  accounted  for  0.58%  (see  Table  2)  of  the  competence  score. 

The  Croiilmtk's  alpha  for  the  Instructional  Technology  Scale  on  the  familiarity 
variuble  was  .W4. 


rum'ivL'd  Hkiiltf  in  thw  UMmiiiutU  ULtimul  dmm  and  technology. 

The  finding  pertinent  to  research  question  4  indicate  a  moderate  extent  of 
perceived  ;      regarding  the  use  nf  Inst  met  iomil  design  and  technology.  All  four 
comparisons  reported  n  menu  value  which  tell  within  the  range  of  moderate  extent  of 
perceived  skill  ( 'nmpnrlsoiis  of  menus  and  medians  revealed  very  little  difference 
between  these  measures  nl  rent  in  I  tendency. 

A  significant  difference  wiih  observed  on  the  scale  assessing  perceived  skill  among 
tlu>  teachers  grouped  by  I;  vcls  of  training  In  instructional  design  and  technology. 
Contributors  to  f  lie  significant  difference  were  observed  between  the  Inservice  and  No- 
Training  groups  and  between  the  f  College  plus  Inservice  and  the  No-Training  groups.  No 
significant  illffcrciiccs  were  observed  on  the  mean  responses  of  teachers  grouped  by  region, 
academic  subject,  and  years  of  teaching  experience. 

An  examination  of  t lie  mean  responses  by  the  four  independent  research  variables 
(flee  Table  1)  showed  that  perceived  skill  regarding  the  use  of  instructional  design  and 
technology  was  highest  mining  teachers  with  College  plus  Inservice  training.  The  lowest 
mean  wiik  among  teachers  with  4   5  years  of  teaching  experience. 

Perceived  skill,  although  not.  correlated  with  competence,  was  highly  correlated  with 
familiarity  and  value.  Perceived  skill  was  observed  to  account  for  2%  of  the  competence 
score  (see  Tuhle  15). 

The  Cmnhuck's  alpha  for  the  Instructional  Technology  Scale  on  the  perceived  skill 
variable  was  .Mil. 


Summary 

Data  analysis  revealed  the  following  major  findings: 

Among  the  four  independent  variables  (place  of  work,  academic  subject,  training  and 
years  of  teaching  experience),  only  training  in  instructional  technology  was  significantly 
related  to  differences  in  competence,  perceived  skill,  familiarity,  and  values  regarding 
instructional  design  and  technology. 


A  significant  difference  wiih  roiiHiHti*nt ly  observed  tor  respondents  with  College  plus 
luservice  training  both  on  tht»  IT  Scale  mi  til  flu*  Competence  'IVhI.  Tin?  No  Training  group 
consistently  ranked  lowest  among  the  training  Kinup"  on  tin*  IT  Scale.  Although  the 
Competence  Test  score  of  the  No  Training  group  was  not  ntatint i<  ally  different,  it  ranked 
second  highlit  topping  those  with  cither  only  College  or  only  liiKervicc  training. 

Then1  wore  very  high  correlations  iinimiK  the  variables  of  perceived  skill,  familiarity, 
mul  values.  On  the  other  hand,  correlations  lii«twoiMi  these  "self  reported"  variables  and 
assessed  competence  in  instructional  design  wore  very  low. 

Trniningin  instructional  technology  wok  tin1  single  iiiiihI  important  predictor  of 
scores  on  the  competence  test,  yet  it  accounted  for  u  relatively  small  amount  of  the  variance 
(12.9%), 

Kcspon dents  reported  moderate  degrees  of  perreiviMl  Hk ill,  fuinilinrity,  and  positive 
value  toward  instructional  design  und  technology;  their  average*  score  on  the  competence  test 
was  H0%, 

Conclusions  and  Recommendations 
The  insults  of  the  study  suggest  the  need  to  interpret  any  "self-reported"  data 
regarding  familiarity  with  or  competence  in  instructional  design  with  extreme  caution. 
Needs  assessment  that  rely  on  self-assessments  may  be  very  inaccurate.  The  results 
suggest  that  the  constructs  of  perceived  skill,  familiarity,  and  valuing  may  be  nearly 
identical    indistinguishable  psychometrically.  Urban/rural  differences,  academic  subject 
"penalty,  and  years  of  teaching  experience  are  not  significantly  related  to  teachers  attitudes 
or  skills  in  instructional  design  and  technology. 

Only  training  in  instructional  technology  was  significantly  related  to  greater 
perceived  skill  in,  familiarity  with,  and  greater  valuing  of  instructional  design  and  technology 
and  increased  competence  in  instructional  design.  Teachers  with  College  plus  Inservice 
training  were  the  only  group  which  had  consistently  high  mean  responses  in  their 
competence,  perceived  skill,  affect,  and  familiarity.  This  suggests  that  both  College  and 
Inservice  t  raining  are  required  to  increase  the  actual  level  of  competence  of  Philippine 
Homndary  school  teachers  regarding  instructional  design  and  technology.  The  finding  which 
indicates  that  the  College  training  and  Inservice  training  groups  performed  no  better  than 
those  with  No  Training  raises  the  issue  of  training  quality  provided  to  both  groups  at  the 
proservice  and  inservice  levels.  As  a  result,  a  review  of  the  preservice  curriculum  relative  to 
tin*  selected  components  of  IT  is  in  order.  Likewise,  the  inservice  training  programs  need  to 
he  carefully  examined, 

The  Competence  Test  of  this  study  emphasized  basic  teaching  competencies,  such  as 
writing  objectives,  developing  test  items,  choosing  teaching  strategies,  etc..  A  mean 
performance  of  60%  seemed  to  suggest  that  whatever  upgrading  of  teaching  competencies 
was  conducted,  the  training  did  not  adequately  provide  the  needed  skills  to  improve  the 
teaching  competencies  of  teachers.  Sixty  percent  performance  on  planning,  implementing, 
and  evaluating  instruction  strongly  implies  a  need  for  training  on  the  use  of  instructional 
design. 

Since  the  data  is  correlational,  causation  cannot  be  confirmed.  However,  this  finding 
is  supportive  of  the  importance  of  training  especially  in  the  instructional  technology  Held. 

Educational  implications  include  the  need  to  review  or  perhaps  refocus  the  present 
preservice  curriculum  and  inservice  training  programs  for  secondary  school  teachers. 
Training  programs  should  be  designed  based  on  actual  needs,  not  on  the  perceived  needs  of 
teachers.  Training  should  focus  on  skills  essential  for  effective  instruction.  The  preservice 
curriculum  should  be  reviewed  in  the  light  of  the  findings  in  this  study. 

Research  implications  include  the  investigation  of  other  variables  which  could 
contribute  to  teachers1  performance.  Other  research  concerns  are  the  determination  of  the 
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extent  to  which  school  administ.  tors,  teaching  schedule,  or  loading  can  contribute  to 
touchers'  high  performance.  There  is  also  a  need  to  replicate  the  Htudy  with  other 
populations.  It  is  apparent  that  there  is  a  lack  of  well  developed,  validated  instruments  to 
measure  competence  in  instructional  design.  The  instrument,  used  in  this  study  could  serve 
as  a  beginning  point  to  develop  such  a  tool. 
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Abstract 


Anthropologist  and  sociologist  have  found  that  technological  innovators  are  more  likely  to 
find  success  when  they:  1)  recognize  the*  group's  unique  identity;  2)  assist  the  group  in 
understanding  and  using  the  technology;  3)  empathize  with  the  group  environmental  and 
cultural  needs,  and  4)  empower  the  group  to  meet  those  needs.  After  reviewing  related 
education  literature,  12  questions  (3  each  lor  4  factors)  were  established  for  considering  how 
articles  in  the  major  research  journals  of  AliCT  (1953.1993)  addressed  issues  that 
demonstrated  an  awareness  of  the  teachers1  culture  in  school,  Of  the  847  articles  reviewed, 
24%  recognizes  the  teachers*  roles,  13%  demonstrated  nn  understanding  of  the  teachers' 
working  conditions,  15%  provide  specific  explanations  of  how  educational  technology  can  be 
integrated  into  school  structures  and  10%  identified  specific  needs  and  corresponding 
educational  technology  solutions  for  empowering  teachers,  Recommendations  include 
greater  emphasis  on  teacher  conditions  and  relevance  in  the  research  literature,  articulation 
of  how  educational  technology  can  be  made  effective  within  the  schools'  culture,  and 
strategies  for  empowering  teachers  through  educational  technology  resources. 


Humans  are  social.  Much  of  our  incentive  lor  learning  comes  from  interacting  with 
and  being  recognized  by  others.  Oncu  the  responsibility  of  parents,  changes  in  the  American 
family  and  society  are  shifting  expectations  for  hocIhI  development  toward  teachers. 
Teachers  shape  the  instruction  and  activities  in  the  classroom  and  they  serve  as  role  models 
for  students.  Many  of  the  crucial  contributions  to  social  development,  including  those 
requiring  mentoring  and  pun  interaction,  am  too  complex  and  situational  to  be  fully 
integrated  within  machine-mediated  instructional  systems  (('lark,  1084;  Kerr,  1989;  Martin 
&  Clemente  1990).  Even  so,  the  systems  employed  hy  educational  technologists  (e.g., 
computers  and  multimedia)  are  playing  increasingly  significant  roles  in  many  aspects  of 
American  life  from  entertainment  to  business  and  education.  These  conditions  call  for 
HtrateglcH  that  capitalize  on  the  potential  of  educational  technology  systems  while  at  the 
same  timo  recognizing  the  unique  contributions  that  teachers  make  in  meeting  the  social 
needs  of  learners, 

Kducntional  technologists  differ  in  their  opinions  regarding  the  teacher's  role  in 
directing  educational  technology.  Some  educators  (Aust,  Allen  &  Bichelmeyer,  1989;  CL,*k, 
10H4;  Clemente  &  Martin  1990;  Kerr,  1986)  believe  that  teachers  are  capable  and  willing  to 
contribute  to  the  advancement  of  educational  technology.  While  others  (Heinich,  1984,  1985) 
believe  that  the  use  of  educational  technology  should  be  directed  by  instructional  systems 
managers  with  teachers  serving  as  components  in  instructional  systems.  Issues  concerning 
who  Hhould  direct  instructional  systems  development  aside,  research  on  change  agents  and 
Hchool  reform  (Finn,  1990;  Frymier,  1987;  Goodlad,  1990;  Grady,  1988;  McDonald,  1989; 
IVarlman,  1988;  Timar  &  Kirp,  1989)  provides  compelling  evidence  that  innovations  have  a 
high  degree  of  failure  when  teachers  do  no  believe  they  are  significantly  involved  in  and 
empowered  by  the  innovation  (Maeroff,  1988). 

To  consider  the  evolution  of  these  and  other  perspectives  regarding  teachers  and 
educational  technology,  we  conducted  an  initial  review  of  literature  in  anthropology, 
sociology  and  education.  This  review  was  used  to  derive  a  series  of  questions  for  reviewing 
H47  articles  that  appeared  in  the  major  research  journals  of  the  Association  of  Educational 
Communication  and  Technology  (AECT)  from  1953  to  1993.  Our  intent  is  to  uncover  trends 
and  insights  for  directing  research  and  development  agendas  that  engage  teachers  in  the  use 
and  design  of  educational  technology. 

Cultural  Awareness 

In  1952  Edward  Spicer  explained  "It  has  become  something  of  a  commonplace  to  say  'People 
resist  change*  but  a  generalization  that  has  more  facts  to  support  it  is  the  opposite  'People 
accept  change1.... "  However,  sociologists  also  note  that  once  a  group  rejects  innovation  the 
chance  that  the  group  will  reject  further  innovation  increases  markedly  (Rogers,  1983). 
What  can  be  done  to  limit  the  risks  of  introducing  technological  innovation  to  teachers? 
Many  anthropologists  (Spicer,  1952;  Wolcott,  1981)  believe  that  the  answers  can  be  found  by 
carefully  considering  the  culture  of  the  receiving  group  before  introducing  technological 
innovation,  Wolcott  (1981)  identified  four  reasons  groups  reject  technological  change: 

1 .  Change  appears  to  threaten  basic  securities. 

2.  The  group  does  not  understand  the  proposed  technology. 

If  The  group  perceive  the  innovators  as  outsiders  and  therefore  resist  the  technology, 

d.  People  do  not  vary  customary  behavior  or  tools  unless  they  have  a  need  for  change. 

Innovation  rejection  is  not  the  only  risk  in  introducing  technological  innovation  to  a  group. 
An  anthropological  study  of  Yir  Yoront,  a  tribe  of  Australian  aborigines,  exemplifies  the 
negative  impact  that  introducing  technology  can  have  on  the  receiving  group's  culture 
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(Blumenfeld,  Hirschbuhl  &  Al-Rubaiy,  1979).  In  the  Yir  Yoront's  isolated  stone-age  culture, 
tho  stone  ax  was  a  "privileged"  symbol  jf  authority  used  in  elaborated  trading  practices 
(Sharp,  1952).  When  missionaries  decided  to  widely  distribute  more  effective  steel  axes,  they 
seriously  disrupted  the  social  structure  and  trmlitioiiH  of  tho  tribe.  Attendance  at  traditional 
meetings  and  social  functions  decreased.  Trade  practices  and  rituals  became  chaotic  and  The 
Yir  Yoront  myths  and  beliefs  were  compromised.  Hoenuso  of  the  missionaries1  haphazard 
innovation  introduction,  tb  j  Yir  Yoront  loss  a  significant  part  of  their  cultural  identity  and 
received  no  long-term  benefits. 

As  defined  in  this  article,  "empowerment  through  technological  innovation"  is  when  a 
group  of  people  addresses  n~eds  or  roceivos  benefits  that  cannot,  be  mot  by  current  strategies 
or  technology.  Furthermore,  a  technology  in  "empowering"  only  when  it  satisfies  this  criteria 
without  imposing  significant  negative  efforts  on  tin*  group's  culture,    The  above  literature 
was  used  in  deriving  four  categories  to  consider  when  introducing  technological  change  into  a 
culture.  Innovators  are  more  likely  to  be  successful  (or  empowering)  wlu*n  they: 
1)  recognize  the  group's  unique  identity;  2)  assist  the  group  in  understanding  and  using  tin* 
technology;  3)  empathize  with  the  group  environmental  and  cult  lira!  nt-eds,  and  <l)  empower 
the  group  to  meet  those  needs. 

Article  Review  Method 

A  total  of  847  articles  appearing  in  the  professional  jour  nil  is  of  the  AKCT  wt-re  reviewed., 
including:  Audio  Visual  Communications  Review  (19H3  •  19(53),  AV  Communications  Review 
(1964  •  1977),  Educational  Communications  and  Technology  Journal  (1978  -  1988)  and 
Educational  Technology  Research  and  Development  (1989-1993).  After  identifying  the  four 
basic  factors  for  introducing  change  into  a  culture,  we  conducted  a  further  review  of 
education  literature  to  derive  the  criteria  for  evaluating  the  AECT  articles.  This  review 
included  educational  technology  studies  that  were  used  to  construct  13  items  for  applying  the 
basic  anthropological  concepts  specifically  to  the  empowerment  of  teachers  througn 
educational  technology.  Three  items  were  developed  as  indicators  for  ot'ch  of  tk'j  foi.r 
factors.  A  question,  concerning  teachers  being  directed  by  instructional  system  design 
professionals,  was  also  added  in  response  to  and  extended  dialog  in  AECT  literature 
(Clemente  &  Martin,  1990;  Heinich,  1984,  1985;  Martin  &  Hemente,  1990;  Shrock  & 
Higgins,  1990). 

A  HyperCard  data  base  wan  developed  that  included  tho  journal  name,  title,  volume, 
number,  authors  and  year  of  publication.  Each  article  appeared  oil  a  separate  eard.  The 
thirteen  items  were  listed  as  buttons.  If  an  article  included  information  that  corresponded  to 
one  of  the  thirteen  items,  tho  appropriate  button  was  clicked  and  the  association  lx»twmMi 
item  and  article  was  automatically  recorded.  Below  are  the  thirteen  items  used  to  ovnluato 
the  articles.  Each  listing  shows  the  question  asked,  the  rationale  for  including  the  item,  the 
criteria  used,  and  an  example  for  satisfying  the  criteria. 

Recognition 

The  following  questions  concern  how  educational  technologists  can  recognize  touch  »ri.  as 
primary  users  of  educational  t  echnology  and  therefore  cause  teacher  to  fool  more  comfortable 
using  the  technological  innovations. 

1:        Describes  tho  toucher's  role? 

Rationale:        Anthropologists  (Wolcott  198!;  Spicer,  1952)  note  that  a  receiving  group 
rejects  a  technology  when  it  threatens  the  group's  basic  securities.  Similarly,  Educational 


Technologists  (Heinich,  1984)  found  that  some  teachers  will  go  so  far  as  to  sabotage 
educational  resources  that  they  find  threatening.  Consequently,  teachers  may  find 
educational  resources  less  threatening,  if  the  developers  recognize  teachers  as  the  receiving 
group  of  the  technology. 

Criteria:         This  item  was  checked  if  the  article  suggested  (even  briefly)  teacher 
involvement  with  the  technology. 

Example:  The  article  mentions  teachers  as  users  or  observers  of  educational  technology. 
2:        Explains  to  teachers  how  to  use  technology? 

Rationale:       Anthropologists  state  that  a  group  of  people  feels  less  threatened  by  a 
technology  if  the  use  of  a  technology  is  communicated  well  (Blum  enfold,  Hirschbuhl  &  Al- 
Rubaiy,  1979).  Correspondingly,  teachers  may  not  accept  technology,  if  they  do  not 
understand  its  use. 

Criteria:  This  item  was  checked  if  teachers'  use  of  the  technology  was  clearly  defined. 

Example:        The  article  describes  how  the  technology  is  used  by  teachers  as  a  record- 
keeping device,  student  evaluation,  content  presentation  and/or  lesson  developing  tool. 

3:        Explains  how  the  technology  will  make  the  teacher  more  effective? 

Rationale:       Anthropologists  found  that  a  group  feels  acknowledged  by  a  technologist,  if 
he/she  explains  how  the  technology  will  make  thetn  more  effective  in  their  livelihoods 
(Rogers,  1983;  Spicer  1952).  Instructional  designers  hIho  must  describe  how  technology  will 
make  teachers  more  effective  (Clemente  &  Martin,  1990). 

Criteria:  This  item  was  checked  if  the  article  explained  how  teachers  use  of  technology 
would  make  them  better  teachers. 

Example:        The  article  describes  why  using  un  instructional  design  could  help  a  teacher 
to  '.each  to  specific  curriculum  objective,  The  article  explains  several  advantages  of  using  an 
overhead  projector  in  the  classroom, 

Eripathy 

Ah  explained  in  both  rduuittnnnl  (Uoodlud,  1990;  Grady,  1988;  McDonald,  1989)  and 
anthropological  (Wolcott,  1IIH1;  Muinunfeld,  Hirschbuhl  &  Al-Rubaiy,  1979)  literature, 
innovators  are  n*n  elved  uh  "outsiders"  unless  they  demonstrate  an  understanding  or 
"empathy  for"  the  group's  culture.  As  lona  as  the  group  perceives  the  technologist  as  an 
outsider,  the  group  will  reject  the  technciogy.  In  like  manner,  teachers  reject  educational 
technology  as  an  outsider's  intervention,  if  the  technologist  does  not  appear  to  empathize 
with  the  "real-life"  culture  and  environment  of  the  teacher. 

4:        Describes  how  available  the  technology  is  for  teachers? 

Rationale:       When  change  agents  attempt  to  introduce  a  technology  that  is  not  currently 
available  to  a  group,  they  also  risk  appearing  as  outsiders  (Blumenfeld,  Hirschbul,  &  Al- 
Rubaly,  1979;  McDonald,  1989).  This  perceived  lack  of  understanding  and  inconvenience  can 
cause  the  receiving  group  to  reject  the  technology.  For  example,  laachers  rejected  early 
educational  television  programming  because  good  programs  were  not  available  during 
convenient  class  times  (Cuban,  1984). 

Criteria:  This  item  was  checked  if  the  article  described  clearly  where  and  how  teachers 
ciin  iruiu  access  to  the  technology 

Example:        The  article  describes  how  a  teacher  or  administrator  can  obtain  a  book  on  a 
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particular  instructional  design.  The  author  explains  how  many  overhead  projectors  are 
currently  available  in  most  school  districts. 

5:        Considers  the  teacher's  workload? 

Rationale:        Anthropologists  suggest  that  innovators  will  be  considered  outsiders,  if  they 
do  not  consider  the  environmental  and  cultural  demands  on  the  group  (Wolcott,  1981). 
Teachers  similarly  are  often  on  tight  schedules  that  limit  their  ability  to  use  technology 
(Bichelmeyer,  1992).  Educational  technology  strategies  are  likely  to  be  more  accepted  if 
educational  techn-ologists  make  an  effort  to  accommodate  or  lessen  the  teacher's  workload. 
Criteria:  This  item  was  checked  if  the  article  mentioned  the  daily  tasks  demanded  by 

teacher?  and  the  time  frame  i.i  which  they  have  to  complete  these  tasks. 
Example:         The  article  describes  how  lesson  plans  are  demanded  by  most  district  and  an 
instructional  design  is  a  simple  method  of  satisfying  these  districts'  demands.  The  author 
mentions  something  about  accommodating  class  schedule.  For  example,  the  average 
classroom  size  in  1992  was  more  than  26  students  and  an  overhead  projector  can  present 
information  easily  to  a  group  of  30  or  more. 

C         Describes  how  educational  technology  improves  teacher/  student  interactions? 

Rational*1:        For  most  groups,  social  interactions  and  interconnectedness  are  the  customs 
of  their  culture.  If  the  innovation  is  perceived  as  threatening  to  a  valued  custom,  the 
innovation  will  not  be  used  (Spicer  1952).  Likewise,  one  of  the  teacher's  most  valued  customs 
is  the  one-on-one  interaction  with  students.  Teachers  cite  the  disruption  to  student-teacher 
interaction  as  a  reason  for  rejecting  some  instructional  system  design  implementation* 
(Martin  &  Clemente,  1990). 

Criteria;        '  This  item  was  checked  if  the  article  explained  how  technology  may  reduce, 
increase  or  change  the  interac-tions  between  teachers  and  students. 
Example:         The  author  describes  how  the  instructional  design  elicits  students  to  ask 
more  questions  of  a  teacher.  The  article  explains  how  the  overhead  projector  increases 
students  to  be  more  reflective  of  their  questions  to  the  teacher. 

Assistance 

Anthropologists  state  a  group  rejects  a  technology  if  it  does  not  assist  them  in  performing 
valued  task  in  their  current  environment.  (Blumenfeld,  Hirschbul,  &  Al-Rubaly,  1979;  Rogers 
1983;  Spicer  195*).  This  assistance  comes  only  after  the  group  understands  the  technology 
thoroughly;  the  technologist  considers  the  strategies  the  technology  is  replacing;  and  explains 
how  the  technology  will  be  managed. 

7:         Explains  teacher  or  pre-teacher  training  procedures? 

Rationale:        Anthropologists  find  that  groups  ignore  technology,  if  it  is  not  proven  to  assist 
the  group  is  some  manner.  If  a  group  contains  a  thorough  understanding  of  the  technology, 
they  are  more  likely  to  recognize  its  helpfulness  (Blumenfeld,  Hirschbul  &  Al-Rubaly,  1979; 
Rogers  1983;  Spicer  1952).  Training  is  one  of  most  effective  means  of  guaranteeing  this 
understanding.  Teachers  are  less  likely  to  adopt  innovation  if  they  believe  that  the  training 
is  inadequate  (McCombs,  1986). 

Criteria:  This  item  was  checked  if  the  article  described  the  process  for  conducting 

training  in  the  use  and  benefits  of  the  technology. 

Kxample:        The  author  mentions  instructional  design  courses  that  are  now  being  offered 
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at  teacher-education  colleges.  The  article  describes  a  successful  workshops  for  training  in 
the  use  of  educational  technology. 

8:        Describes  the  relationship  to  existing  strategies  and  structures? 

Rationale:        In  anthropology,  a  group  ignores  an  innovation  if  the  traditional  methods  of 
the  group  are  not  acknowledged  and  protected.  In  addition,  change  will  not  be  accepted  if 
advantages  over  traditional  strategies  are  not  proven  and  observable  (Blumenfeld, 
Hirschbuhl  &  Al-Rubaiy,  1979  Rogers,  1983;  Wolcott,  1981).  Teachers  will  reject 
replacement  technology,  if  it  is  not  proven  more  helpful  than  the  methods  they  are  currently 
using  (Martin  &  Clemente,  1990). 

Criteria:  This  item  was  checked  if  the  article  described  how  the  technology  will 

augment  current  teaching  practices,  or  describes  how  the  technology  will  prove  more  effective 
than  current  teaching  methods. 

Examples:       The  author  explains  how  the  instructional  design  does  not  replace  the  control 
and  flexibility  of  the  teachers  in  directing  instruction.  The  article  explains  how  teachers  can 
use  an  overhead  projector  more  effectively  and  explains  how  teachers  might  replace  some 
blackboard  functions  with  an  overhead  projector. 

9:         Explains  approaches  for  managing  and  organizing  media? 

Rationale:  A  technology  may  not  be  perceive  as  helpful  if  the  effort  involved  in  managing 
and  organizing  its  use  is  unreasonable.  Teachers  do  not  want  to  become  overburdened  by  the 
bureaucracy  of  instructional  management  (Kerr,  1989). 

Criteria:  This  item  was  checked  if  the  article  described  who  is  responsible  for  or  how 

the  technology  will  be  stored,  organized,  maintained  and/or  evaluated. 
Example:         The  author  explains  who  should  be  responsible  for  the  maintenance  and 
repair  of  overhead  projector  or  the  evaluation  of  an  instructional  design. 

Empowers 

Many  anthropologists  feel  that  the  purpose  of  any  technology  is  to  empower  the  group. 
Empowerment  (as  used  here)  means  not  only  acceptance  of  the  technology,  but  personalizing 
it  ho  (»nch  individual  in  the  group  can  modify  the  technology  to  varying  situations. 

1 0:       Explains  how  the  teachers  can  select,  control  or  modify  technology? 

Rationale:        Anthropologists  believe  that,  especially  during  the  early  stages,  user  should 
he  offered  considerable  control  and  practice  in  implementing  technology  (Rogers  1983). 
Teachers  express  that  they  are  willing  to  become  involved  in  the  implementation  of 
innovations  (Aust,  Allen  &  Bichelmeyer,  1989). 

Criteria:  This  item  was  checked  if  the  article  described  the  procedures  for  a  teacher  to 

une  in  modifying  or  selecting  technology  to  fit  their  personal  class-room  environment. 
Exam  pie:         The  author  explains  how  a  teacher  can  use  an  overhead  projector  in  varying 
decrees  of  classroom  light.  The  article  describes  how  to  select  an  instructional  design  that 
ran  ho  used  for  both  lesson  and  long-range  course  planning. 

1 1 ,       Describes  strategies  for  administrators  to  support  teacher  use  of  technology? 

Rationale:        Anthropologists  have  found  that  individuals  in  a  group  will  adopt  the 
innovation  more  rapidly  if  the  adoption  power  is  njoije  'equally  distributed'  (Freeman,  Azadi, 
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1983).  Teachers  as  well,  may  reject  a  techn  ology  unless  control  of  the  technology  is  given 
directly  to  teachers  not  the  school  administration  (Cuban,  1984). 

Criteria:  This  item  was  checked  if  the  article  explained  how  administrators  can  he 

involved  in  distributing  the  control  of  educational  resources  to  teachers. 

Example:         The  author  explains  how  the  use  of  educational  technology  can  be  controlled 

by  teachers.  The  article  describes  how  teachers  are  involved  in  evaluating  an  instructional 

design. 

12:       Describes  teacher's  involvement  in  the  innovation  and  planning? 

Rationale:        Studies  in  anthropology  (Spicer  1952;  Rogers,  1983)  explain  that  the  chance 
for  empowerment  through  a  new  technology  increases  if  the  receiving  groups  are  closely 
involved  in  the  development  cycle.  Teachers  moreover,  are  resistant  to  reforms  that  rely  on 
"outside"  change  agents  unless  they  have  are  sufficiently  involved  in  the  reform  (McDonald 
1989;  Sirotnik  &  Clark,  1988;  Timar  &  Kirp  1989). 

Criteria:  This  item  was  checked  if  the  article  described  how  teachers  are  involved 

directly  or  indirectly  in  the  development  of  the  technology. 

Example:         The  author  explains  how  the  complaints  of  blackboard  dust  by  teachers  were 
considered  in  the  innovation  of  overhead  projectors.  The  article  describes  how  an 
instructional  design  was  developed  by  a  committee  consisting  of  secondary  school  teachers. 

Teachers  in  Roles  Secondary  to  Instructional  System  Design  (ISD)  Specialists. 

13.       Casts  teacher  as  inflexible  or  in  a  secondary  role? 

Rationale:        Studies  in  both  educational  technology  and  anthropology  have  note  that 
many  technologists  blame  the  receiving  group  when  a  technology  fails  (Wolcott,  1981).  Such 
conceptions  may  increase  chances  that  future  technology  introductions  will  be  rejected. 
Criteria:  This  item  was  checked  if  the  article  described  the  teachers1  function  in  terms 

of  an  overall  plan  under  the  direction  of  an  instructional  system  designer  or  administrator. 
This  item  was  also  checked  if  the  article  described  teachers  as  unresponsive  or  inflexible. 
Example:         The  article  describes  the  teacher  as  a  monitor  in  a  fully-mediated  classroom. 
The  author  explains  that  instructional  planning  should  be  performed  exclusively  by 
professional  instructional  designers. 

Review  of  AECT  Journals 

Analysis  of  the  data  collected  from  the  847  articles  appearing  in  AECT  journals  was  intended 
to  reveal  the  relative  emphasis  placed  on  factors  relating  to  teacher's  involvement  in 
educational  technology.  Hypertalk  routines  were  developed  to  automatically  tabulate  the 
data  entered  by  the  reviewer.  Results  are  shown  as  percent  of  articles  that  met  criteria  for 
each  item.  Charts  were  constructed  using  Microsoft  Excel. 

Limitations: 

This  is  an  exploratory  investigation  designed  to  gain  general  insights  and  refine 
procedures.  Efforts  were  made  to  maintain  objectivity  in  reviewing  all  articles.  However, 
the  review  of  the  articles  was  conducted  by  one  reviewer,  and  inter-rater  reliability  has  not 
been  established.  Once  inter-rater  reliability  is  established,  more  involved  statistical 
procedures  including  correlation  across  factors  may  be  warranted. 


Table  1 

Percent  of  Articles  that  Address  Teacher  Related  Issues  in 
AECT  Research  Journals  (1953  .  1993) 


Recognition  26% 

1:  Describes  the  teachers  role?  24 

2:  Explains  to  teachers  how  to  use  technology?  14 

3:  Explains  how  the  technology  will  make  the  teacher  more  effective?  17 

Empathy  13% 

4:  Describes  how  available  the  technology  is  for  teachers?  4 
5:  Considers  the  teacher's  workload?  5 

6:  Describes  how  educational  technology  improves  teacher/student  interactions? 
12 

Assistance  14% 

7:  Explains  teacher  or  pre-teacher  training  procedures?  5 

8:  Describes  the  relationship  to  existing  strategies  and  structures?  12 

9:  Explains  approaches  for  managing  and  organizing  media?  5 

Empower  10% 

10:  Explains  how  the  teachers  can  select,  control  or  modify  technology?  8 

1 1 :  Describes  how  administrators  can  support  teacher  use  of  technology?  1 

12:  Describes  teacher's  involvement  in  the  innovation  and  planning?  5 

13:  Casts  teacher  as  inflexible  or  in  a  secondary  role?  3 

Note:  874  articles  were  analyzed 

This  table  shows  the  percent  of  articles  that  met  the  criteria  for  each  item.  Factor  headings 
(e.g.  Recognition)  show  the  percent  of  articles  that  satisfied  criteria  for  one  or  all  of  the  three 
items  associated  with  that  factor. 
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Figure  1 

Teachers  Recognized  and  Teachers  as  Secondary  to  ISD 
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Figure  1  shows  the  percent  of  the  847  AKCT  articles  (by  year)  that  satisfied  the  criteria  for 
the  Teacher  Recognition  nnd  Teacher  as  Secondary  to  ISD  factors.  If  and  article  was  deemed 
iis  satisfying  the  Teacher  Recognition  criteria,  the  reviewer  must  have  answered  yes  to  at 
least  one  on  the  following  three  questions:  This  article  describes  the  teachers  role?  (and/or) 
This  article  explains  to  teachers  how  to  use  technology?  (and/or)  This  article  explains  how  the 
technology  will  make  the  teacher  more  effective?  To  satisfy  the  Teacher  as  Secondary  to  ISD 
criteria  the  reviewer  must  have  answered  yes  to  the  question:  This  article  casts  teacher  as 
inflexible  or  in  a  secondary  role?  Note  that  the  peaks  in  the  Teacher  Recognition  factor 
appear  slightly  after  the  peaks  in  the  Teacher  as  Secondary  to  ISD  factor. 
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Figure  2  shows  the  percent  of  the  847  AECT  articles  (by  year)  that  satisfied  the  criteria  for 
the  Teacher  Empathy  factor.  If  an  article  was  deemed  to  meet  this  criteria,  the  reviewer  had 
answered  yes  to  at  leant  one  on  the  following  three  questions:  This  article  describes  how 
available  the  technology  is  for  teachers?  (anchor)  This  article  considers  the  teacher's 
workload?  (and/or)  Thin  article  iIchitUh'H  how  educutional  technology  can  improve 
teacher/student  interaetionH? 


Figure  3 

Describes  Teacher  Assistance 
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Figure  3  shows  the  percent  of  the  847  AECT  articles  (by  year)  that  satisfied  the  criteria  for 
the  Teacher  Assistance  factor.  If  an  article  was  deemed  to  meet  this  criteria,  the  reviewer 
had  answered  yes  to  at  least  one  on  the  following  three  questions;  This  article  explains 
teacher  or  pre-teacher  training  procedures?  (anchor)  This  article  describes  the  relationship  to 
existing  strategies  and  structures?  (and/or)  Explains  approaches  for  managing  and 
organizing  media? 
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Figure  4 

Teacher  Empowerment 
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Figure  I  lihoWH  I  hi*  iinrriMit  of  llu*  847  AECT  articles  (by  year)  that  satisfied  the  criteria  for 
I  In*  Tnii'lul  KuipnWMrninit  factor.  If  an  article  was  deemed  to  meet  this  criteria,  the 
jf*VM<wt<r  ) i j ut  answered  yes  to  at  least  one  on  the  fallowing  three  questions:  This  article 
cxplmriH  how  I     h'arherw  ran  neluct,  control  or  modify  technology?  (and/or)  This  article 
(Ii'hitiIh'h  Imw  fiilinlniHlriiturH  ejjn  .support  teacher  use  of  technology?  (and/or)  This  article 
(Ifwriliiw  h'in  tiri  M  involvement  in  the  innovation  and  planning? 

Much  of  the  lltiTjit  uri%  in  t  he  AKCT  research  journala  focuses  on  the  effects  of  mediating 
learning  with  technology.  About  u  quarter  of  the  articles  (24%)  do  at  least  recognize  the 
tear  hern'  roles  (see  Tnblc  1).  However,  the  other  factors  concerning  teachers*  conditions  in 
ncIiooIh,  have1  not  received  substantial  attention  hi  educational  technology  research  and 
development.  Only  about  1H%  of  the  articles  mentioned  issues  that  demonstrate  an 
understanding  of  the  teachers1  working  conditions  in  schools,  Fewer  than  15%  provide 
specific  explanations  of  how  educational  technology  can  be  integrated  within  existing 
curriculum  or  related  school  structures.  Finally,  about  10%  identified  specific  needs  and 
corresponding  educational  technology  solutions  for  empowering  teachers. 

As  the  Yir  Yoront  discovered,  the  desirability  of  technology  is  relative  to  cultural 
values.  Teachers  are  the  first  line  receivers  of  educational  technology  and  this  review 
indicates  that  educational  technologist  may  not  be  paying  enough  attention  to  the  teachers' 
culture.  One  risk  is  that  educational  technologist  will  appear  as  outsiders  who,  even  with 
superior  technology,  will  be  summarily  ignored  by  many  teachers.  A  more  consequential 
concern  is  that  zealous  efforts  to  widely  demonstrate  educational  technologies  without 
considering  cultural  implications,  may  be  disruptive  of  both  the  culture  within  schools  and 
society  at  large. 

The  review  of  the  AECT  articles  also  reveals  patterns  in  the  amount  of  attention  paid 
to  teacher  issues.  Figures  1-  4  show  somewhat  tandem  cycles  with  the  four  teacher  factors 
(recognition,  empathy,  assistance  and  empowerment)  peaking  in  the  early  1960s  and  mid 
1980s.  We  do  not  have  a  single  compelling  explanation  for  these  cycles.  Note  in  Figure  1 
that  the  peaks  in  the  percent  of  articles  that  cast  teachers  as  inflexible  or  in  secondary  roles 
appear  just  ahead  of  the  peaks  in  teacher  recognition.  One  explanation  is  that  the  peaks  and 
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valleys  represent  reaction  to  specific  events  that  question  sensitive  issues  related  to  either 
teachers  or  technology,  Fur  example,  Itohert  Heinich  published  and  article  in  1984 
explaining  that  teachers  roiild  ho  c  misidcred  hb  components  in  ISD.  Several  authors 
(CMc-iiiiMitci  &  Martin,  1MHI;  Hoiuich,  1HH4,  1085;  Martin  &  Clemente,  1090;  Shrock  & 
lliggins,  100(1)  followed  with  articles  that  cited  Heinich  and  often  presented  contrary 
positions.  Possibly,  llt'inirir*  article  spawned  this  extended  debate  hut  there  may  have  been 
any  number  of  reasons  for  the  rise  in  nttention  paid  to  teacher  issues,  The  cycles  might  he  a 
reaction  to  national  agendaa  such  as  the  reform  movement,  the  Holmes  group  movement, 
federally  fundeil  initiatives  or  ptv  My  they  reflect  the  preferences  of  the  editorial  boards. 
More  in  depth  reviews  of  the  lite*  aire  are  needed  to  bettor  understand  and  interpret  these 
cycles, 

lieeiaiiiiiendatioiiH 

The  most  far  reaching  means  for  addressing  teacher  issues  in  educational  technology  tuny  be 
a  comprehensive  plan  for  encouraging  research  authors  to  relate  findings  to  the  needs  of 
teachers.  This  would  not  require  extensive  changes  in  tho  research  designs  or  compromises 
in  essential  basic  research.  Instead,  authors  would  more  explicitly  describe  how  their 
(  Hinting  research  relates  either  directly  to  the  needs  of  teachers  or  how  the  research  could 
lead  to  better  understanding  of  teaching  and  learning.  Of  course,  AK<  <T  has  oilier  literature 
(e.g.,  Tech  Trends)  that  is  intended  to  describe  advances  in  and  practical  Uses  of  educational 
technology.  However,  the  notion  that  there  are  different  "levels"  of  literal  urn  for  different 
levels  of  audiences  promote  a  counter  productive  isolationist  image  that  deters  teacher 
involvement.  A  more  reasoned  approach  is  to  provide  different  forms  of  literature  with 
different  purposes  that  continue  to  maintain  broad  audionce  appeal. 

Much  of  the  current  research  and  development  concerning  educational  technology 
focuses  on  strategies  for  perfecting  the  interaction  between  learners  and  machines.  Research 
addressing  such  issues  us  feedback,  graphics,  hypertext,  integrated  learning  systems, 
cognitive  styles,  learner  control  and  pacing,  is  needed  to  assist  designers  in  refining 
instruction.  However,  such  research  should  not  overshadow  questions  concerning  teacher 
empowerment  with  educational  technology.  Do  teachers  consider  their  culture  as  different 
and  separate  from  the  educational  technologists  -  what  are  the  differences?  How  can 
machines  improve  the  interaction  between  students  and  teachers?  Is  educational  technology 
meeting  the  teachers'  real-world  needs  for  curricula  and  other  support?  Which  teaching 
activities  are  best  handled  by  technology  and  which  are  best  left  to  teachers?  How  can 
administrators  best  serve  to  empower  teacher  with  educational  technology? 

Developers  of  educational  technology  can  address  teacher  empowerment  by  providing 
greater  teacher  control  of  tho  characteristics  of  educational  resources.  For  example, 
computer  instructional  programs  can  provide  overriding  teacher  control  of  feature  such  as: 
level  of  difficulty,  amount  of  remediation,  time,  pacing,  graphics  and  audio.  Teachers 
(possibly  in  consultation  with  students)  could  then  determine  the  most  appropriate 
combination  according  to  t  heir  understanding  of  the  learner,  the  curriculum  and  current 
classroom  conditions.  In  this  way  the  resource  is  optimized  through  the  teachers  unique 
skills  in  mediating  the  complex  learning  situations  and  the  mentor  role  of  the  teacher  is 
maintained.  Developers  should  also  address  teacher  needs  by  providing  guides  that  explain 
such  concerns  as  the  process  skills  addressed,  required  prerequisites,  intended  grade  level 
rind  consistency  with  curricula  . 

Kducatinmit  technology  can  empower  teachers  by  incorporating  social  enrichment 
activities  within  instructional  systems.  The  <  iper  Woodbury  series  developed  at 
Vanderbildt  University  provides  an  example  of  this  form  nf  empowerment.  Hovcral  of  the 
Jasper  Woodbury  segments  begin  by  using  video  to  establish  a  problem  Helving  scenario  and 
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then  encourage  students  to  break  into  groups  for  data  analysis  and  forming  hypotheses. 
During  these  group  discussions  the  students  have  an  opportunity  to  interact  with  peers  while 
the  teachers  role  in  moderating  the  group  and  directing  learning  is  maintained. 

Advances  in  computer  networks  can  also  be  used  to  address  teacher  empowerment 
through  improved  communications  among  teachers  and  community  publishing  of  information 
services.  Computer  and  networking  technologies  are  advancing  rapidly  as  government 
officials  and  private  industry  seek  to  establish  a  National  Information  Infrastructure  (Nil). 
This  is  a  critical  period  for  educators  to  become  involved  in  demonstrating  ways  the  Nil  can 
empower  teachers  and  learners.  One  such  strategy  is  through  the  development  of  national 
conferencing  capabilities  for  a  variety  of  teaching  specialties.  Using  currently  available 
technology  this  would  involve  establishing  special  interest  bulletin  boards  and  providing 
broad  accessibility  to  schools  so  that  teachers  that  share  common  goals  but  are  separated  by 
geographical  and  scheduling  barriers  can  easily  exchange  ideas  for  improving  teaching.  As 
the  networking  and  multi-media  technologies  advance,  teachers  will  be  able  to  demonstrate 
and  monitor  the  on-going  work  of  exemplary  teaching. 

Another  example  of  network  technology  empowering  teachers  is  the  community 
publishing  of  network-based  educational  resources  (see  for  example;  Aust,  1093,  1992).  Must 
teachers  have  a  repertoire  of  successful  teaching  skills  but  neither  the  time  or  support  for 
publishing  extended  collections.  Network  information  services  are  capable  of  automating  the 
processes  of  contributing  and  organizing  information  so  a  teacher  can  contribute  a  single 
resource  (lesson  plan,  lab  activity  or  field  trip  description)  that  becomes  part  of  a  large 
resource  data  base  that  may  eventually  he  easily  searched  from  any  school  in  the  nation.  In 
this  way,  the  teacher  will  contribute  to  libraries  of  educational  resources  while  becoming 
vested  in  the  goal  of  empowering  all  teachers. 
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Abstract 


Publication  is  at  the  heart  of  scholarship.  Traditionally,  print  media  has  been 
perhaps  the  most  important  vehicle  for  communicating  research  results  and  scholarly 
thought.  Experimentation  in  publishing  electronic  journals  (ejournals)  began  in  the  past  two 
decades,  both  in  North  American  and  Europe.  This  paper  explores,  from  a  scholarship 
perspective,  the  rationale  for  publishing  electronic  journals.  We  will  examine  why  scholars 
publish,  the  advantages  and  disadvantages  of  various  media,  and  the  issues  of  credibility, 
accessibility,  and  permanence  within  an  ever-changing  technical  environment.  Interpersonal 
Computing  and  Technology:  An  Electronic  Journal  for  the  21st  Century  (IPCT-J)  will  be  used 
as  an  example  of  a  scholarly,  peer- reviewed,  electronic  journal. 


Publisher  a  of  electronic  journals  need  to  be  first  and  foremost  concerned  with 
that  which  every  other  publisher  of  scholarly  journals  is-credibility,  access,  and 
permanence-all  within  an  every  changing  technical  environment. 


INTRODUCTION 

Publication  is  at  the  heart  of  scholarship  (Cohen,  1993):  part  and  parcel  of  academic 
life  and  function.  Tradit;cally,  print  media  has  been  perhaps  the  most  important  vehicle 
for  communicating  research  results  and  scholarly  thought. 

Electronic  Publishing 

Experimentation  in  publishing  electronic  journals  (ejournals)  began  in  the  past  two 
decades,  both  in  North  American  and  Europe.  While  there  have  been  many  models  of 
ejournals  over  the  years,  it  has  only  been  recently  that  more  than  a  handful  have  survived 
on  non -commercial  networks  (e.g.,  other  than  networks  such  as  DIALOG;  Prodigy)^,  In 
early  1993,  there  were  probably  less  than  three  dozen  such  journals  in  the  humanities  that 
arc1  strictly  non  commercial  and  electronic,  and  perhaps  double  that,  if  the  hard  sciences  are 
included.  Some  ^journal  publishers  are  non-profit,  university  publishers.  However,  hardly 
any  of  the  electronic  journals  (ejournals)  are  peer  refereed.  As  the  number  of  ejournals 
increases,  there  is  a  need  for  understanding -for  development  and  research  concerning  this 
new  vehicle  for  the  dissemination  of  scholarly  work. 

This  paper  will  explore,  from  a  scholarship  perspective,  the  rationale  for  publishing 
electronic  journals.  We  will  examine  why  scholars  publish,  the  advantages  and 
disadvantages  of  various  media,  and  the  issues  of  credibility,  accessibility,  and  permanence. 
Interpersonal  Computing  and  Technology:  An  Electronic  Journal  for  ihe  21st  Century  (1PCTJ) 
will  be  used  as  an  example?  of  a  scholarly,  peer-reviewed,  electronic  journal. 

Testing  the  Waters 

In  1992,  promoting  the  use  of  technology  in  the  classroom  was  one  of  the  goals  of  the 
Center  for  Teaching  and  Technology  at  Georgetown  University's  Academic  Computer  Center. 
One  avenue  towards  this  goal  was  to  bring  scholars  from  around  the  world  together  to 
discuss  ideas,  to  tap  their  expertise,  and  to  communicate  research  results  so  that  everyone 
can  learn. 

Eventually,  I  thought  a  scholarly  journal  might  be  one  way  to  accomplish  a  part  of 
this  goal.  Since  the  Center  for  Teaching  and  Technology  is  part  of  the  Academic  Computer 
Center,  an  journal  in  electronic  form  seemed  a  natural.  Still,  before  launching  a  journal  in 
any  format,  many  decisions  needed  to  he  made.  In  order  to  gather  some  marketing  data,  I 
formed  the  Interpersonal  Computing  and  Technology  Discussion  list  (IPCT-L),  hoping  that 
the  list  would  be  valuuhlu  in  determining  if  a  market  existed  for  a  new  journal,  to  find 
potential  authors  and  subscribers,  find  to  locate  potential  editors  and  reviewers. 

I  intended  that  1PCT-Ii  should  focus  on  the  uses  of  instructional  technology,  hut, 
from  the  first,  the  list  attracted  a  number  of  communications  scholars  interested  in 
computer-mediated  communication,  amongst  whom  was  Dr.  Gerry  Phillips,  Professor 
Emeritus  of  Speech  Communication,  The  PnmiHylvaiiiu  Htato  University. 

Phillips  and  I  entered  into  early  discussions  miicertiing  tin  proposed  journal.  In  lau> 
summer  of  1992  we  decided  to  proceed  with  tint  journal  Phillips  esffrrnil  his  considerable 
experience  and  editorial  services  as  Editor  in  Thief  and  wn  asked  Mauri  Collins,  a  doctoral 
student  in  Instructional  Hy stums  at  Putin  Htate,  (and  the  other  IPCT-L  listowner  and 
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moderator),  to  assume  the  responsibilities  of  Managing  Editor,  and  join  is  to  form  the  first 
editorial  board,  with  Mark  Evangelista  providing  technical  support  at  Georgetown,  and 
ukmik  computer  time  and  storage  space  provided  by  both  the  Academic  Computer  Center  and 
Information  Systems  at  Georgetown  and  the  Center  for  Academic  Computing  at  Penn  State. 


SCHOLARSHIP 

Why  do  scholars  publish? 

It  is  important  to  understand  something  about  scholarship  before  going  too  much 
further  into  the  details  of  publishing  and  editing  an  electronic  journal.  Scholars  basically 
author  for  two  reasons,  the  first  being,  to  communicate  their  research  and  ideas  to  others. 
As  a  by-product  of  the  peer  recognition  from  this,  promotion,  tenure  and  academic  or 
professional  honors  are  derived  from  their  contributions  to  the  field  (Smith,  1991). 

The  Life-Cycle  of  Scholarly  Thought 

How  does  scholarship  progress?  Where  do  ideas  come  from?  How  do  they  move  from 
ideas  to  published  articles?  Researchers  and  scholars  generate  ideas  through  such  activities 
as  observing,  discussing,  and  reflecting.  Alter  this  brainstorming  or  formation  step,  the 
person  often  drafts  the  idea  or  hypothesis  and  at  some  point  decides  to  circulate  a  draft  to 
colleagues.  This  can  be  done  through  conference  presentations  or  the  mail,  fax  or 
electronically  in  the  prepublication  stage.  If  the  material  is  favorably  received,  the  author 
may  then  submit  the  work  to  a  journal  for  peer  review.  Passing  muster  here  results  in  the 
article  being  published.  Some  typical  postpublication  steps  include  dissemination  to 
individuals  and  libraries,  indexing,  and  abstracting.  Once  other  scholars  react  to  the  work 
with  their  feedback,  new  ideas  are  shaped  or  new  research  hypotheses  generated  and  the 
cycle  continues. 


[INSERT  FIGURE  "SCHOLARSHIP"  ABOUT  HERE] 
(See  Appendix  A  for  figure  "Scholarship) 

What  is  wrong  with  this  model? 

I  don't  think  there  is  anything  wrong  with  the  model  per  se.  Yet  there  are  factors  in 
the  current  implementation  of  the  print  journal  model  that,  when  not  adequately  addressed, 
result  in  inefficiencies  and  ineffectiveness,  posing  a  threat  to  the  scholarly  process  as  we 
move  toward  the  21st  century. 

Consider,  for  example,  library  acquisitions.  With  books  and  journals  going  out  of 
print  quickly,  and  interlibrary  loan  often  not  meeting  the  needs  of  scholars,  libraries  are 
inclined  to  acquire  as  many  printed  materials  as  they  can  afford.  In  business  this  has  been 
described  as  the  "just-in»case"  model  of  inventory.  Acquisitions  are  made  just  in  case 
someone  needs  them  (quickly).  Compare  this  to  a  just-in-time  model  where  journal  articles 
are  available  online  to  scholars  at  the  library  or  at  their  own  workstation  only  whim  and  if 
they  are  needed. 

Lack  of  Timely  Feedback 

Part  of  the  scholarship  cycle  involves  timely  feedback  to  the  author.  If  one  is  working 
on  AIDS  research,  any  delay  in  moving  scholarly  endeavors  forward  could  mean  life  or  death- 
-literally.  Yet  delayed  feedback  to  a  historian  seems  much  less  time  sensitive.  As  an 
example  of  the  typical  time  lag  in  print  jmvnals  lots  look  at  the  article  1  wrote  in  1990, 
informally  circulated  to  a  few  people  during  the  beginning  of  1991.  1  submitted  it  to  a 
educational  technology  journal  in  October  1991.  It  was  peer-reviewed  and  accepted  for 
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publication  in  October  1992.  Finally  it  was  printed  and  distributed  late  in  1993-2.5  or  3 
years  after  I  wrote  it.  I  may  now  be  able  to  remember  the  title,  but  even  I  can't  remember 
the  main  points. 

Formal  publication  doesn't  complete  the  cycle.  Readers  need  to  receive  a  copy  of  the 
article,  either  through  the  library  or  individually.  Then  they  have  to  respond.  If  the  reader 
chooses  to  comment  to  me  directly  the  response  time  may  be  relatively  short.  Howevert 
should  he  or  she  choose  to  send  their  response  for  peer  review  to  the  journal,  or  even  as  a  a 
Letter  to  the  Editor,  it  might  take  from  one  to  three  years  for  that  feedback  to  reach  me. 

This  lack  of  interactivity  in  the  print  process  is  becoming  more  and  more  a  fatal  flaw 
in  the  scholarship  cycle.  The  original  author's  thinking  has  moved  on  from  what  was  being 
articulated  those  many  years  previously.  Regardless  of  whether  the  subject  matter  was 
history  or  biology  or  literature,  had  there  been  more  timely  feedback,  new  ideas  that  are  now 
lost  could  have  been  generated  (Harnad,  1991).  This  is  why  scholars,  especially  in  the 
natural  sciences,  mathematics  and  technology,  do  not  rely  on  print  journals  to  keep  current 
in  their  field. 

Costs 

The  cost  of  scholarly  journals  is  rising  at  a  rate  much  greater  than  the  rate  of 
inflation  or  of  libraries'  budgets  and  funding  sources.  There  is  little  by  way  of  cost  savings 
when  moving  from  print  to  electronic  form  regarding  editing  or  peer  review.  It  should  be 
noted  that  the  cost  of  reviewing,  editing,  and  even  the  production  costs  associated  with 
electronic  journals  are  often  "hidden"  because  of  the  volunteer  or  subsidized  labor  of  editors, 
reviewers  and  publishers.  I  do  think  there  are  some  significant  cost  savings  involved  in 
ejournal  production,  and  especially  delivery  (e.g.,  no  postage  costs;  no  paper  and  printing 
costs  to  the  publisher).  These  costs  savings  can  be  passed  to  the  subscribers. 

Limits  on  the  Size  of  Print  Journals 

There  are  many  economic  factors  that  put  pressure  on  print  journals  to  limit  the 
number  of  pages  published,  causing  editors  and  publishers  to  lower  the  amount  of  scholarly 
work  accepted.  Additionally,  there  are  tendencies  to  reject  such  things  as  research  showing 
no  significant  differences,  and  feedback  from  readers  such  as  Letters  to  the  Editor  lose  their 
places  to  articles  reporting  original  research  results. 

ADVANTAGES  OF  ELECTRONIC  JOURNALS 
There  are  the  obvious  advantages  such  as  the  lower  costs  for  electronic  production 
and  distribution.  Secondly,  the  acceptance  rate  for  articles  is  not  limited  by  economic  factors 
(this  assumes  there  is  a  pool  of  scholarly  work  that  was  rejected  for  economic  reasons  and 
not  because  of  lower  quality  than  that  accepted  for  the  paper  journal).  Some  other  significant 
factors  are  discussed  below. 

Increased  Speed 

There  is  no  question  that  the  production  and  distribution  steps  during  publication 
can  he  sped  up  by  moving  to  epublishing.  Regarding  the  peer  review  process,  there  can  be 
gains  in  speed,  but  not  if  the  reviewers  let  the  electronic  copies  sit  on  their  disk,  as  is  often 
the  case  with  papers  for  review  sitting  on  their  desks.  But  it  is  at  the  feedback  link  in 
postpublication  that  most  benefit  could  be  derived  from  epublications.  Metz  (1991)  points 
out  that  a  community  of  scholars  can  he  more  dynamic  and  interactive  in  their  commentary 
to  authors  in  a  way  that  is  currently  impossible  with  paper  journals. 
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Collaboration 

Much  of  the  idea  above  regarding  speed  dovetails  with  the  notion  of  building 
knowledge  collaboratively.  Electronic  media  allows  more  interactivity  among  scholars  within 
and  between  disciplines.  The  potential  exists  not  only  for  more  collaborative,  international 
effort,  but  also  for  dissemination  and  scholarship  among  authors  in  different  fields  of  study. 
The  electronic  network  is  able  to  provide,  as  Hamad  (1992)  points  out,  cross-disciplinary 
review  and  discussion  of  research. 

Different  Models 

Morris  (1989)  characterizes  the  impact  of  technology  in  three  stages;  First, 
technology  is  used  to  automate  that  which  has  been  done.  Secondly,  tasks  change  because 
we  can  do  things  we  couldn't  do  before.  Finally,  society  itself  changes.  Langenberg  (1989) 
tells  us  that  computers  and  telecommunications  are  merging  to  form  information  tecnnology. 
That  is,  the  power  of  computers  to  store  and  manipulate  data,  along  with  the  speed  and 
convenience  afforded  by  telecommunications  to  transfer  data,  equals  an  unprecedented 
ability  to  handle  information  using  technology. 

So,  what  impact  will  this  emerging  information  technology  have  upon  society- -in  this 
case  on  scholarly  society?  King  (1991)  states: 

Just  as  radio  offered  possibilities  unimagined  by  the  newspaperman,  and 
just  as  television  offered  possibilities  unimagined  by  the  radioman,  so  the 
electronic  network  medium  will  provide  opportunities  for  accessing  and  using 
information  which  so  far  have  not  been  imagined  by  the  print  mind.  (p.  6) 

While  much  of  what  we  have  been  doing  in  epublisbing  mimics  traditional  printed 
journals,  there  are  those  persons  who  are  anxious  to  realize  new  and  different  models  in 
epublishing.  IPCT  Journal  pretty  much  by  design,  uses  the  same  model  as  print  journals. 
However,  there  are  those,  especially  in  the  sciences,  who  are  working  with  ejournals  which 
clearly  publish  new  images,  such  as  3-dimensional  models,  that  could  not  be  done  with  print 
technology.  As  sound  and  animation  are  more  commonly  used,  many  exciting  things  will 
appear  in  journals  that  were  impossible  to  do  before  epublishing.  Similarly,  software  will 
allow  sophisticated  searching  strategies  and  linking  of  current  issues  to  back  issues  or  to 
other  journals-in  ways  impossible  with  print  technology.  Finally,  there  are  authors  such  as 
Hamad  (1990)  who  see  "scholarly  skywriting"  using  electronic  media  as  significantly 
changing  the  scholarly  society. 

One  of  the  keys  to  many  of  these  advantages  is  the  fact  that  epublishing  is  free  from 
time  and  space  limitations.  Distributions  and  correspondence  throughout  the  scholarship 
cycle  are  not  limited  by  time  or  geography.  Review,  distribution,  feedback  and  submissions 
can  take  place  in  an  instant  from  around  the  corner  or  across  the  world. 


PROBLEMS  WITH  EJOURNALS 

While  ejournals  facilitate  communications  among  ncholnrs,  they  are  not  n  panacea. 
However,  1  believe  many  of  the  problems  ejournals  face  are  overstated  and  overvalued. 
Mention  is  often  made  of  the  increased  possibilities  for  plagiarinm,  or  the  question  raised  of 
the  credibility  of  a  scholarly  journal  that  vies  for  space  with  electronic  junk  mail  or 
sometimes  does  not  even  have  enough  structure  to  have  an  ISSN. 

There  are  still  some  issues  that  are  significant  and  which  will  not  easily  be 
overcome.  Some  are  faced  by  any  (new)  journal  regardless  of  the  media  in  which  it  is 
published;  other  problems  arc  unique  to  the  electronic  medium.  For  instance,  there  an; 
other  activities  which  fulfill  scholars'  need  to  interact  (quickly)  with  colleagues.  While  it  is 


expensive  to  travel,  conventions,  seminars,  workshops,  and  newsletters  all  provide  scholars 
with  opportunities  to  respond  to  one  anothers'  work,  and  to  set  forth  their  own  research  and 
ideas. 

Another  example  has  to  do  with  the  downside  to  one  of  the  advantages  to  electronic 
publishing  (epublishing)  mentioned  above.  While  publishers  and  editors  are  not  limited  in 
the  number  of  pages  they  can  published  due  to  cost,  a  higher  acceptance  rate  often  tends  to 
give  the  mistaken  perception  of  lower  prestige  to  the  ejournal. 

But  as  many  authors  note  (e.g.,  Okerson,  1991;  Harnad,  1990;  Amiran  et  al.,  1991; 
Shamp,  1992;  Bailey,  1992;  Manoff,  1992),  perhaps  the  biggest  problems  faced  by  ejournals 
are  those  faced  by  any  (new)  journal.  John  Franks  (1993)  speaks  of  the  primary  functions  of 
publishers  as  credentialing,  archiving  and  marketing.  I  prefer  to  think  of  my  function  as 
publisher  as  ensuring  credibility,  accessibility,  and  permanence  (Harrison,  1991).  Further, 
all  these  functions  are  within  a  constantly  changing  technical  environment. 

Credibility 

Without  scholars'  submissions  or  without  readers,  there  is  no  journal.  The  higher 
the  credibility  of  the  journal,  the  higher  the  quality  of  submissions  received.  There  simply 
may  not  be  enough  peer  interaction  surrounding  any  new  journal  to  generate  the  authorship 
and  readerships  necessary.  Additionally,  an  ejournal  is  often  not  considered  an  official 
university  activity.  This  often  lessens  the  perception  of  credibility  in  scholars1  minds.  But 
the  biggest  threat  to  an  ejournal's  credibility  is  that  most  promotion  and  tenure  committees 
do  not  count  an  article  in  ejournals  equally  (or  at  allP  compared  to  that  same  article  in  a 
print  journal. 

There  are  at  least  two  ways  to  work  toward  overcoming  problems  with  credibility. 
The  quality  control  aspect  of  journals  is  derived  from  the  peer-review  process.  An  absolute 
must  therefore,  in  building  a  journal's  credibility  is  to  have  a  top  notch  editorial  board  and 
the  best  referee*.  IPCT  Journal  has  a  world-class  group  of  editors  and  associate  editors 
(their  main  function  is  reviewing  submissions)  and  submitted  articles  are  held  to  the  highest 
standards  of  peer  review. 

The  problem  of  promotion  and  tenure  committees  not  giving  authors  credit  for  work 
done  in  ejournals  may  have  to  wait  until  administrators  and  3enior  faculty  on  promotion 
and  tenure  committees  retire.  As  the  new  cohort  of  leaders  move  into  these  positions,  they 
may  be  more  familiar  with  ejournals  and  their  merits. 

Accessibility 

It  doesn't  matter  if  one  has  the  vest  scholarly  journal  in  the  world  if  no  one  knows 
that  it  exits  or  how  to  uccchh  it.  Again,  the  cost  and  economics  of  epublishing  could  change. 
The  netwnrkud-basod  ejnurnalH  have  been  able  to  gain  a  foothold  because  networks  like 
Hitnet  and  the  Internet  are  heavily  subsidized.  If  these  networks  lose  their  subsidies  and 
become  commercialized,  the  economies  may  change  drastically  and  seriously  affect  the  access 
by  users  to  ejournals  (Bailey,  1992). 

Potential  subscribers  often  have  major  technical  difficulties  with  equipment,  or 
getting  training  and  other  information  to  access  the  internet,  let  alone  in  finding  ejournals. 
Secondly,  users  often  need  training  on  the  internet  resources  (the  electronic  highway  is  there, 
hut  not  everyone  knows  how  to  use  it).  Even  if  the  potential  subscriber  has  the  equipment, 
and  the  knowledge  to  use  the  system,  finding  out  about  a  particular  journal  of  interest,  or 
even  if  one  exists,  is  still  a  daunting  task. 

Even  the  standards  for  citation  of  articles  published  by  ejournals  are  in  their  infantry. 
While  some  authorities  (Tuttle,  1991)  see  citing  ejournals  as  just  like  any  other  publication, 
others,  including  IPCT  Journal's  Managing  Editor,  suggest  that  information  'needs  to  be 
included  in  ihe  citation  regarding  a  document's  primary  archive  site. 
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Another  problem  ejournals  have  in  that  very  few  are  abstracted  or  indexed  through 
common,  services.  To  a  large  extent,  scholar**'  awareness  of  a  particular  article  germane  to 
their  own  interests  and  work  is  dependent  upon  searching  for  an  abstract  in  standard 
sources.  Until  there  is  a  critical  mass  of  significant  ejournals  (probably  meaning  sufficient 
demand  from  customers),  indexers  and  abstraction  services  are  unlikely  to  include  ejournals. 

The  publisher  and  editorial  board  spend  a  good  deal  of  time  discussing  copyright. 
From  a  scholar's  perspective,  communicating  their  ideas  and  research  findings  to  the  largest 
audience  possible  is  one  of  their  chief  goals  (as  long  as  the  proper  attribution  is  given  to  their 
work).  Yet  publishers  who  market  and  distribute  print  journals  have  a  more  limited  and 
limiting  concept  of  "audience"  as  those  who  will  pay  for  subscriptions.  Since  scholars  almost 
always  transfer  copyright  to  the  publisher  before  publication,  a  conflict  of  purposes  arises 
between  the  commercial  goal  of  the  publisher  and  the  communications  purpose  of  the 
scholar. 

Ejournals,  for  the  most  part,  are  more  aligned  with  the  scholar's  point  of  view.  To 
the  scholar,  access  to  ideas  supersedes  ownership  of  ideas  (Metz,  1991).  Most  ejournals, 
including  IPCT  Journal,  ask  authors  only  for  first  publishing  rights  rather  than  exclusive 
rights.  IPCT  Journal  allows  authors  to  retain  control  over  subsequent  individual  use  of 
their  work,  as  long  as  mention  is  made  in  reprints  that,  the  article  first  appeared  in  IPCT 
Journal.  IPCT  Journal  retains  copyright  to  the  compilation,  and  can  grant  to  libraries,  etc. 
the  rights  to  use  the  journal  in  its  entirety. 

Another  topic  that  the  editors  and  publisher  discussed  at  length  was  the 
distribution  format.  There  are  many  models  that  ejournals  use.  Some  ejournals  distribute 
each  article  as  soon  as  it  has  been  finally  accepted.  Some  require  the  journal  subscriber  to 
be  a  member  of  a  related  discussion  list.  IPCT  Journal  started  that  way  but  opened  a 
separate  "journal  only"  list-so  that  subscribers  Huch  as  libraries  could  receive  the  journal's 
table  of  contents  without  the  conversation  generated  on  the  discussion  list.  Others  send  out 
a  group  of  articles,  including  a  table  of  contents  and  the  full  text.  At  IPCT  Journal,  we  have 
chosen  to  send  out  a  Table  of  Contents  (see  Appendix  1),  with  instructions  on  how  to  retrieve 
the  full  text  for  each  article  via  Listserv,  gopher  or  FTP.  Our  primary  motivation  for  this 
decision  was  that  scholars  read  articles,  not  journals.  Other  considerations  include  the  size 
limitations  on  some  email  accounts.  IPCT  has  over  IfiOO  subscribers  from  42  countries  in 
February,  1994.  Some  subscribers,  especially  in  countries  outside  North  America,  have 
severe  limitations  regarding  input  to  their  email  account.  Other  subscribers  nro  on 
commercial  networks  (e.g.,  Prodigy;  CompuServe)  and  an*  ehurged  for  ouch  piece  of  mail  they 
receive  (depending  more  or  less  also  on  the  length  of  ouch  pion*  of  mail). 

Some  ejournals  will  also  mail  out  paper  copies  of  each  issue  or  of  all  articles  in  a 
year,  and  some  disseminate  via  CD-ROM,  floppy  disk  and  fax.  Kneh  has  their  unique 
advantages  and  disadvantages,  and  many  would  work  for  libraries  and  individuals  not  on 
the  internet.  IPCT  Journal  has  been  distributed  only  via  the  internet.  Itoqiiosts  for  other 
forms  of  distribution  will  be  decided  upon  on  a  case  by  rase  basis. 

While  we  can  discuss  gophers  and  FTP  and  Listserv  and  World  Widu  Web,  all  of 
which  are  important  to  access,  the  biggest  effort  to  overcome  the  problem  of  accessibility  is  in 
our  efforts  to  have  ejournals  abstracted  through  common  sources.  While  it  is  important  that, 
persons  in  the  internet  have  access  to  ejournals,  it  is  even  more  important  that  all  scholars 
know  of  the  literature  being  built  and  the  research  being  done,  where  it  is  stored  and  how  to 
retrieve  it  (even  if  just  from  the  library  or  other  workstation). 

One  way  to  find  out  about  ejournals  is  to  subscribe  to  the  related  discussion  lists. 
For  instance,  if  you  an?  interested  or  working  in  distance  education,  there  are  many 
discussion  lists  in  that  field.  Subscribing  to  those  electronic  conferences  is  one  way  to  find 
out  about  ejournals  in  that  discipline  (e.g.,  The  Online  Journal  of  Distance  Education).  In 


the  case  of  IPCT  Journal,  it  was  conceived  on  the  IPCT-L  discussion  list  and  is  regularly 
mentioned  there. 

Finally,  the  acceptance  and  accessibility  of  ejournals  will  greatly  improve  when 
scholars  are  able  to  search  for  them  from  their  workstation,  in  an  easy  and  reliable  way. 

Permanence 

Authors  are  extremely  concerned,  and  rightly  so,  with  the  ephemeral  and  transitory 
perception  of  electronic  text.  No  scholar  wants  to  place  his  or  her  work  in  a  journal  that  is 
destined  to  disappear  or  die.  Ejournals  are  no  exception-in  fact,  if  pasthistory  is  any 
indication,  ejournals  do  not  have  a  good  track  record  with  regard  to  permanence. 

Whether  an  ejournal  ceases  publication  or  not,  their  archives  location  is  often 
problematic.  This  is  one  area  where  libraries  can  offer  a  great  service  by  taking  over  the 
archiving  of  ejournals  from  computer  centers.  The  mission  of  computer  centers  is  clearly  not 
to  preserve  and  maintain  scholarship,  whereas  a  case  could  be  made  that  this  is  a  function 
of  (some)  libraries  (Bailey,  1992).  There  may  be  a  consortium  of  libraries  or  publishers  that 
emerge  over  the  next  couple  years  to  take  on  this  role,  too. 

Technical  Aspects 

It  doesn't  matter  if  you  publish  the  best  journal  in  the  world  if  readers  find  it  too 
hard  to  read.  Ease  of  readability  in  this  media  has  a  significant  bearing  on  the  format  in 
which  journals  are  published,  (e.g.,  PostScript,  LaTeX,  SGML).  IPCT  Journal  is  published 
only  in  ASCII  text.  This  limits  the  graphics,  tables,  half-tones,  mathematical  and  scientific 
notations  that  can  be  used  within  its  articles.  That  works  fine  for  now  for  the  disciplines 
(the  humanities)  which  are  the  source  of  most  articles  that  are  accepted  for  IPCT  Journal. 
But  it  would  not  work  within  many  scientific  or  technical  disciplines. 

Additionally,  there  are  issues  such  as  increasing  file  size  across  the  Internet  and  the 
limited  email  account  space  for  some  users,  that  cause  network  performance  problems.  In 
this  formative  period,  there  is  a  general  lack  of  user-friendly,  "reader"  software  and  lack  of 
sophisticated  browsing  capabilities. 

Some  scholars  argue  (Metz,  1991),  that  the  fluid  and  changeable  nature  of  ejournals 
is  one  of  their  most  powerful  assets.  The  ability  to  alter  and  modify  the  work  based  on 
interaction  and  feedback  to  the  author  is  seen  as  a  real  advantage.  But  this  advantage  also 
creates  a  concern  for  the  security  of  ejournal  text.  While  we  have  an  "official"  copy  of  IPCT 
Journal  issues  on  the  gopher  at  GUVM,  once  distributed,  there  is  no  way  to  know  whether 
versions  of  the  journal's  articles  that  have  been  unofficially  modified  show  up  elsewhere. 

CONCLUSIONS 

Even  if  ejournals  are  less  expensive,  easier  to  access,  or  Have  time,  thiH  is  not  Kointf 
to  allow  ejournals  to  compete  with  print  journals  if  the  ronton!  of  the  ojouriuil  in  tint 
perceived  to  be  of  the  highest  quality  (Willis,  1991).  Having  a  world-clans  group  nf  editors 
and  reviewers,  along  with  the  highest  standards  for  peer  review,  are  necessary  ingredients 
for  the  credibility  of  any  journal. 

Scholars  seek  to  communicate  with  their  peers,  Kjnurnuls  are  especially  efficient  at 
this-provided  most  of  those  peers  are  fluent  in  using  the  internet  ,  Scholars  also  seek  to 
derive  credit  toward  promotion  and  tenure  with  their  scholarly  activities,  ^journals 
currently  do  not  fare  as  well  in  this  regard.  Publishers  and  editors  need  tn  pay  particular 
attention  to  organizational  acceptance  as  a  critical  factor  in  the  Huccess  of  electronic  scholarly 
publishing. 

IPCT  Journal  strives  to  build  its  credibility,  permanence,  and  accessibility  in  much 
the  same  way  a  new  print  journal  does.  This  is  done  within  a  technical  environment  that 
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poses  now  challenges,  yet  offers  vast  opportunities.  Still,  the  functions  of  journal  publishing 
and  editing  remain  the  same  in  many  significant  ways.  It  is  the  earnest  desire  of  the 
publisher  and  editors  of  IPCT  Journal  that  we  contribute  to  scholarly  communication  in 
way«  meaningful  to  our  authors  and  readers. 


Notes 

1  Parts  of  this  paper  are  modified  from  an  article  to  appear  later  this  year  in  the  Journal  of 

the  American  Society  for  Information  Science. 

2  I  am  mukinjj  a  distinction  here  between  an  electronic  journal,  which  is  published  in  only 

electronic  form,  and  an  online  version  of  a  (parallel)  print  journal  (e.g.,  an  electronic 
version  of  the  Harvard  Business  Review  available  through  an  online  service). 
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1.  Letter  from  the  Publisher 

Dear  I MCT  Journal  Headers, 

This  issue  of  Interpersonal  Computing  and  Technology:  An 
Kleetronic  (Journal  for  the  21st  Century,  marks  the  beginning  of 
our  sec  ond  year  of  publication.  It  is  with  a  renewed  sense  of 
our  moving  scholarship  forward  that  the  Editors  and  Publisher  of  IPCT 
Journal  enter  this  new  year.  I  would  like  to  say  just  a  little 
about  our  perceptions  of  the  challenges  of  publishing  an  electronic 
journal  at  this  time  in  history. 

First  and  foremost,  scholarly  journals  need  top-quality 
submissions,  Without  that,  no  amount  of  marketing  or  persuasion  is 
going  to  cajole  busy  people  into  reading  the  articles.  Regardless  of 
the  longevity  of  a  journal,  or  the  medium  in  which  it  is  published, 
people  involved  with  publication  are  constantly  aware  of  at  least  three 
characteristics  that  measure  their  long-term  viability  and  success: 
credibility,  access,  and  permanence. 
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The  most  significant  aspect  nf  rriulihility  mini's  from  the 
uiti<Kriiv  ofthe  peer  review  process.  In  this  regard,  the 
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professional  met  its  that  distinguish  the  Keillors  and  reviewers  nf 
it  inuruul  mi'  perhaps  the  single  bi^^tsst  factor.  I PCT  Journal 
has  a  world  class  and  international  ((roup  of  editors  and  associate 
editors  who  have  diligently  reviewed  submissions  to  the  highest 
standards. 

It  is  my  belief  that  the  publishers  of  electronic  journals 
(eiournals),  need  to  do  everything  possible  in  the  areas  of  ensuring 
access  and  permanence.  With  libraries,  computer  and  technology 
facilities,  and  various  consortia  all  working  together  with  ejoiirnal 
publishers,  the  problems  involving  lack  of  access  and  perinaneure  will 
he  solved  sometime  in  the  future  Until  then,  ejournal  publishers  do 
what  they  can  to  increase  across  to  their  journals,  and  to  lessen  fears 
uuthors  have  regarding  the  lion- permanence  of  ejouruals. 

At  one  level,  it  is  the  ever  changing  technological  environment 
that  ejouruals  exist  m  that  has  an  impact  upon  access  and  permanence 
For  example,  earlier  this  mouth  we  announced  the  availability  of  li't'T 
Journal  via  gopher,  (see  instruction*  under  Retrieval  below)  This 
certainly  increases  network  uhith'  ucccnh  to  the  Journal   Yet  it  does 
nothing  for  increasing  the  awareness  of  IPCT  Journal  in  the  largest  group 
of  persons  who  may  find  the  articles  useful  those  persons  not  on 
the  Internet. 

There  are  a  couple  of  problems  that  arc  unique  to  ejouruals  and 
that  hinder  the  establishment  of  their  credibility   For  example, 
ejournals  are  rarely  abstracted  and  indexed  on  mainstream  services. 
In  seeking  high  quality  submissions,  the  biggest  obstacle  currently  is 
that  most  promotion  and  tenure  committees  don't  recognize  articles 
published  in  ejournals  equally  (or  at  all!)  as  those  in  a  print  journal. 
The  solution  to  this  problem  may  have  to  wait  until  the  group  of 
administrators  and  people  on  T&P  committees  changes  to  persons 
more  familiar  with  technology  and  having  increased  understanding  of 
ejournals. 

Several  traditional  measures  of  a  journal's  value  are  not  available 
to  electronic  publishers  on  the  Internet.  For  instance,  people  ask  me 
"how  many  subscribers  does  your  journal  have?"  The  response  to  even  a 
simple  question  like  that  is  confounded.  Yes,  I  can  say  that  there  are 
1514  electronic  addresses  on  the  IPCT  J@aUVM.aEOROETOWN.EDU  subscription 
list.  But  a  significant  number  of  those  subscribers  are  local  NetNews 
servers,  There  is  no  way  to  tell  how  many  times  the  IPCT  Journal  is 
exploded  on  these  local  servers,  nor  how  often  the  Table  of  Contents  of 
an  issue  is  cross-posted  to  lists  other  than  IPCT-J  and  IPCT-L,  nor  how 


many  times  an  issue  or  individual  articles  are  accessed  via  gopher  or 
from  the  LISTSERV  at  GUVM. 

Page  3 

It  is  with  sincere  appreciation  I  thank  all  the  Editors  for  a 
job  well  done  this  past  year.  We  will  continue  to  bring  the  best 
articles  we  can  find  to  our  readers  in  future  issues.  We  again 
solicit  your  submissions.  Please  let  me  know  what  you  find  useful, 
what  you  would  like  to  see  more  or  less  of,  and  how  the  Journal  can 
he  improved. 

Starting  with  this  issue  we  will  be  including  page  numbers. 
This  should  help  when  articles  in  the  Journal  are  quoted  and  cited. 
Because  we  have  no  way  of  knowing  how  the  pages  will  be  formatted 
on  screen  (or  when  printed)  on  your  particular  system,  we  have 
arbitrarily  set  the  page  length  at  50  lines.  Each  journal  will 
ho  numbered  from  Page  1,  starting  from  the  beginning  of  the  Table 
of  Contents.  Page  numbers  for  all  articles  will  appear  on  the  Table 
of  Contents  and  again  in  the  heading  of  each  article. 

Turning  to  this  issue  of  IPCT  Journal,  a  new  and  noteworthy 
feature  is  a  Book  Review  by  John  Laurence  Miller  of  Seymour 
Papert's  book,  ^Turning  the  Computer  into  the  Children's  Machine  . 
I  think  you  will  find  it  interesting  and  the  beginning  of  many  useful 
hook  reviews  to  come  in  the  Journal. 

Regards, 

Zane  L.  Berge,  Publisher 
BERGE@GUVAX.ACC.GEORGETOWN.EDU 


2.  Retrieval  Instructions  for  Articles 
GOPHER 

IPCT  Journal,  including  all  hack  issues,  is  available  via  gopher  from 
GUVM.CCF.GEORGETOWN.EDU  (or  141.1H1.71.1).  Point  your  gopher  to  this 
location  (port  70)  and  select  from  the  top  menu,  "LISTSERV  maintained 
Files  and  Notelogs/."  Alternatively,  coming  in  via  Gopher  menus,  from 
"Other  Gopher  Sites"  or  "International  Gopher  Networks."  follow  the  menus 
down:  North  America/USA/Washington  D.C./Georgetown  University/Information 
Systems/Listserv  maintained  Files  and  Notelogs.  (Note:  The  IPCT-L 
Discussion  List  Notelogs  can  be  found  here,  too.) 

LISTSERV 

Articles  are  stored  as  files  at  LISTSERV&GUVM.BITnet.  To  retrieve 
a  file  interactively,  send  the  GET  command  appearing  both  before  and  after 
the  article  abstract  to  L1STSERV&GUVM. 

To  retrieve  the  article  as  a  e-mail  message  add  F=MAIL  to  your 
interactive  message,  or  send  an  e-mail  note  in  the  following  format: 
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To:listsery@guvm.georgetown.ed  • 


GET  <FILENAME>  IPCTV2N1 

The  GET  command  GET  IPCTV2N 1  PACKAGE  will  retrieve  the  entire  issue. 
Page  4 

[WARNING:  This  wilt  send  all  7  files  with  a  total  of  over  3100  lines  ] 

The  listserv's  Internet  nddross  is  LlSTSERV@GUVM.GEORGETOWN.EDU 

Back  issues  of  the  journul  are  Htored  nt  L1STSERV@GUVM.  To  obtain  a 
list  of  all  available  files,  send  the  following  message  to 

L1STSERV&GUVM:  INDEX  IPCTJ.  The  name  of  each  issue's  table  of  contents 
file  begins  with  the  word  "CONTENTS". 

FTP 

1PCT-J  articles  can  he  retrieved  by  FTP  (File  Transfer  Protocol.  FTP  to 
GUVM.CCF.GEORGETOWN.EDU  or  141.161.71.1,  logon  IPCT  J,  password  is  GUEST. 
All  1PCT-J  files  are  currently  archived  in  ASCII  format  only. 

If  you  experience  difficulties  with  these  instructions,  please  consult 
your  local  site  administrator  for  specific  instructions  that  may  apply 
to  your  system. 


3.  Contents 

THE  NEW  BRUNSWICK  NET:  THE  2 1ST  CENTURY  NOW 
Rory  McGreal,  Tele  Education  New  Brunswick,  Canada 
To  retrieve  this  article  GET  MCGREAL  IPCTV2N1 
ABSTRACT 

The  New  Brunswick  Net:  The  21st  Century  Now 

The  province  of  New  Brunswick,  Canada  is  implementing  one  big 
distributed  electronic  highway  for  the  use  of  the  public  and 
private  sectors  in  every  part  of  the  province.  The  strategic  drivers 
for  this  action  are  the  desire  to  attract  high  technology  companies; 
to  provide  local  companies  with  a  competitive  advantage;  to  improve 
government  services  and  increase  revenues  while  cutting  costs;  and  to 
extend  education,  healthcare,  justice  and  other  government  services. 
Stakeholders  identified  are:  the  provincial  and  federal  governments, 
private  sector  companies  including  the  telco  and  cable  companies  and 
local  communities.  New  Brunswick  has  an  advanced  fully  digital  fiber 
optic  infrastructure  that  gives  it  a  competitive  advantage  over  other 


45 


9 

ERIC 


states  and  provinces,  allowing  it  to  implement  21st  century  technology 
province-wide  now. 

Lines:  573 

Pnge  numbers:  1 1-21 

Page  5 

To  retrieve  this  article  GET  MCGKEAL  IPCTV2N1 


AN  EVALUATION  OF  THE  ELECTRONIC  CLASSROOM:  THE  AT&T  TEACHING 
THEATRE  AT  THE  UNIVERSITY  OF  MARYLAND 

Kent  L.  Norman  and  Leslie  E.  Carter,  The  University  of  Maryland 

To  retrieve  this  article  (JET  NORMAN  IPCTV2N1 

ABSTRACT 

This  report  summarizes  reaetions  to  the  AT&T  Teaching  Theater 
at  the  University  of  Maryland.  The  AT&T  Teaching  Theater  is 
an  electronic  classroom  outfitted  with  a  high  performance 
workstation  at  each  student  desk  and  the  instructorUs  podium. 
Networking  and  video  switching  allow  for  interactive 
communication,  file  sharing  distributed  control,  and 
collaborative  learning  experiences.  In  addition,  large 
screen  audio/visual  displays  are  integrated  with  the  system 
to  allow  for  smooth  transitions  from  one  presentation  to 
another.  During  the  Fall  Semester  of  1991  six  different 
courses  were  taught  in  the  classroom.  Instructors  were  asked 
to  relate  their  best  and  worst  experiences  in  the  room  in 
order  determine  what  works,  what  doesnUt,  and  what  needed  to 
be  changed.  In  addition  to  the  use  of  the  room  for  computing 
and  audio/visual  events,  the  best  uses  of  the  room  included 
sharing  of  student  work  on  the  large  screen  monitors, 
collaborative  note  building,  student  polling,  and 
collaborative  problem  solving.  The  major  drawbacks  had  to  do 
with  the  need  for  a  more  seamless  flow  of  events,  the  need 
for  pedagogical  examples  of  how  best  to  integrate  technology 
and  instruction,  ackward  and  complex  connectivity  to  out  of 
class  computer  facilities,  problems  with  computer  software 
and  hardware,  and  problems  with  room  architecture. 

Lines:  876 

Page  numbers:  22-39 

To  retrieve  this  article  GET  NORMAN  IPCTV2N1 
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TEACHING  LANGUAGES  WITH  NKTNKWH 


Terri  Cononelos,  Even  Start  Literary  Program,  Halt  Lake  City,  UT 
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This  paper  describes  an  attempt  at  integrating  computer  network  resources 
into  language*  teaching.  The  case  of  Italian  401-1  taught  in  Spring  1992  at 
the  University  of  Utah  by  Maurizio  Oliva  is  discussed.  In  this  course 
students  improved  their  language  skills  by  interacting  with  native 
speakers.  The  communication  took  place  especially  through  postings  on  the 
Usenet  newsgroup  Soc. Culture. Italian,  and  hy  Email. 
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ABSTRACT 

The  peer-review  journal  is  typically  accepted  as  the  most 
reliable  format  for  information  exchange  in  modern 
societies.  Office  automation  now  makes  possible  a  reduction 
in  delays  associated  with  peer  review.  Advances  in  computer 
technology  have  also  made  possible  the  routine  use  of 
cryptographic  procedures,  including  secure  pseudonymous 
communication  and  secret  sharing  methods,  that  can 
safeguard  porsonal  integrity  and  improve  adherence  to 
procedural  norms.  These  developments  increase  the 
applicability  and  importance  of  the  journal  as  an  authority 
structure,  thereby  making  intra-organizational  journals 
increasingly  attractive.  Such  journals  can  also  play  a 
crucial  role  in  combating  threats  to  tho  organization 
exacerbated  by  the  hew  information  technologies.  In  the 
well  functioning  organisation,  information  distribution 
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tends  to  dominate  decision  making,  therefore  telematic 
enhancements  of  this  function  can  play  an  important  role  in 
organizational  control. 
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ABSTRACT 

Exchanging  notes  on  the  Internet  is  a  "great  equalizer."  People  of 
all  types,  with  all  sorts  of  qualifications,  can  come  together  in 
discussions  on  a  common  ground.  Presumably,  this  would  mean  we  could 
concentrate  on  the  message  without  "considering"  the  source.  But 
people  still  keep  requesting  credentials  and  asking  for  qualifications, 
and  the  issue  of  anonymity  in  contributions  often  becomes  a  real 
issue  in  the  arguments. 

This  concern  about  credentials  and  qualifications  echoes  loudly  in  current 
political  affairs.  Recent  responses  to  President  Clinton's  programs  have 
featured  ad  hominem  attacks  by  various  television  commentators 
and  comedians. 

Bemoaning  this  state  of  affairs,  Frank  Walters,  a  network  contributor  to 
the  LISTSERV  discussion  list,  Clinton«'*Marist,  expressed  his  thoughts  on 
the  elements  of  rhetorical  credibility,  equally  applicable  to  face  to  face 
or  virtual  encounters. 

Lines:  206 

Page  numbers:  64-68 

To  retrieve  this  article  GET  WALTERS  IPCTV2N1 


BOOK  REVIEW 

TURNING  THE  COMPUTER  INTO  THE  CHILDREN'S  MACHINE 
John  Laurence  Miller,  York  University,  Toronto,  Canada 


48 


To  retrieve  this  article  GET  MILLER  IPCTV2N1 
Page  8 
ABSTRACT 

Over  the  last  two  decades,  Seymour  Papert  has  become  known, 
not  only  for  his  pioneering  work  with  the  programming  language, 
Ijogo,  hut  also  as  one  of  the  most  important,  and  most  innovative, 
thinkers  in  the  entire  field  of  education.  His  new  book  The 
Children's  Machine  represents  in  essence  a  refinement  and  an 
elaboration  of  ideas  presented  in  earlier  works,  rather  than  a 
significant  departure.  The  most  important  nev-  idea,  the  concept 
of  constructionism,  offers  an  attempt  to  explain  the  role  of 
concrete  action,  in  connection  with  reflective  thought,  in  the 
emergence  of  new  forms  of  thought  and  new  cognitive  structures. 
Illustrative  examples  are  taken  from  a  wide  range  of  contexts  and 
domains  of  knowledge,  from  cooking  and  horticulture  to 
cybernetics  and  elementary  mathematics. 
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For  the  last  couple  of  years,  Teachers  College,  Columbia  University  and  the  Pulton 
School  (  an  independent  school  in  Now  York  ( -iiy)  have  collaborated  on  the  Dalton 
Technology  Project.   This  project  aims  to  use  networked  multimedia  workstations  to 
produce)  an  environment  that  supports  Htudunt  studying  in  groups  using  authentic  material 
and  contextH.  This  approach  to  education  contrasts  sharply  with  the  usual  approach  which 
has  students  working  individually  to  passively  receive  knowledge  from  teachers  and 
textbooks  using  artificial  problems.  The  project  shares  many  features  with  the  developing 
constructivist  approaches  to  instructional  design  (e.g.,  Jonassen,  1991;  Bednar, 
Cunningham,  Duffy  and  Perry,  1991;  Collins,  Brown  and  Newman,  1990; 
Cognition  and  Technology  Group  at  Vanderbilt,  1990;  Spiro,  Feltovich,  Jacobson  and 
Coulson,  1991),  but  it  differs  from  them  in  emphasizing  design  for  study  as  opposed  to 
design  for  instruction.  Thus,  we  strive  to  create  Ma  place  for  study  in  a  world  of  instruction" 
(McClintock,  1971). 

Seven  Principles  of  Study  Design 

In  addition  to  developing  the  particular  study  systems  for  different  subject  areas  in 
the  Dalton  Technology  Project,  we  have  been  trying  to  specify  what  the  underlying  design 
principles  are  for  such  an  approach.  In  doing  this  we  draw  inspiration  both  from  Cognitive 
Science  (e.g.,  Brown,  Collins  and  Duguid,  1989)  and  from  hermeneutic  interpretation  theory 
(e.g.,  Palmer.  1969).  From  this  effort,  we  have  come  up  with  the  following  seven  study 
system  design  principles: 

1.  Text:  Present  students  with  particular  cultural  objects  (events,  writings, 
images,  artifacts,  scores,  observations,  experiments,  etc.),  the  origin  and 
meaning  of  which  will  confront  them  as  obscure,  a  challenge 

to  the  understanding. 

2.  Context:  Provide  students  with  open-ended  access  to  contextual  materials 

that  may  help  to  clarify  and  interpret  the  cultural  objects  presented  to 
them  and  provide  pathways  leading  from  the  particular  object  to  the 
comprehensive  assemblage  of  pertinent  materials.  On  the  one  hand,  the 
context  must  be  immediate,  and  on  the  other  hand  it  should  include 
everything. 

3.  Engagement:  Situate  the  presentation  of  the  text  and  context  - 

both  the  challenging  cultural  objects  and  their  contextualizing  resources 
in  such  a  way  that  students  will  grasp  strong  ownership  of  the  on-going 
effort  to  interpret  the  material. 

4.  Cooperation:  Have  students  collaborate  in  their  quest  for  interpretative 

understanding,  learning  to  empathize  with  the  interpretative  actions 
of  their  peers. 

5*  Inclusivity:  Use  cognu--  *  apprenticeship  to  show  students  how  to  enlarge 
the  scope  and  power  of  the  contextual  materials  they  bring  to  bear  on 
interpreting  the  text,  moving  the  interpretation  toward  that  ideal 
condition  in  which  all  significant  contextualizing  materials  have  been 
taken  into  account. 


6.  Abstraction:  Encourage  students  to  bring  significant  contexts  to 

bear  upon  multiple,  different  cultural  objects  to  prepare  them  to  transfer 
their  interpretative  skills  to  novel  problems. 

7.  Diversity:  Encourage  students  to  situate  complex  cultural  objects  in  many 
different  significant  contexts  to  prepare  them  to  develop  the  cognitive 
flexibility  of  understanding  things  from  many  points  of  view. 

An  example  program  will  serve  to  illustrate  these  principles,  then  we  will  discuss 
how  to  assess  student  understanding  and  learning  in  these  kinds  of  study  environments.  In 
the  Archaeotype  program,  students  study  ancient  Greek  and  Roman  history  by  using 
observations  of  simulated  archaeological  digs  to  construct  interpretations  of  the  history  of 
these  sites,  while  drawing  upon  a  wide  variety  of  background  information.  The  Archaeotype 
program  (implemented  in  Supercard  on  Macintosh  computers),  which  is  the  earliest  and 
most  fully-developed  of  the  Dalton  Technology  Project  programs,  presents  the  students  with 
a  graphic  simulation  of  an  archaeological  site,  then  the  students  study  the  history  of  the  site 
through  simulated  digging  up  of  artifacts  (the  text),  making  various  measurements  of  the 
artifacts  in  a  simulated  laboratory,  and  relating  the  objects  to  what  is  already  known  using 
a  wide  variety  of  reference  materials  (the  context).  The  students  work  cooperatively  in 
groups,  while  the  teacher  models  how  to  deal  with  such  a  site  then  fades  their  involvement 
while  coaching  and  supporting  the  students  in  their  own  study  efforts  (inclusivity).  The 
students  develop  ownership  cf  their  work  by  developing  their  own  interpretations  of  the 
history  of  the  site  and  musteving  various  kinds  of  evidence  for  their  conclusions 
(engagement).  By  arguinp  with  the  other  students  and  studying  related  interpretations  in 
the  historical  literature,  they  get  a  sense  of  other  perspectives  (diversity).  By  going  through 
the  process  a  number  of  times  bringing  each  contextual  background  to  bear  on  a  number  of 
differnt  artifacts,  the  students  learn  and  understand  the  general  principles  behind  what 
they  are  doing  (abstraction). 

Assessing  Student  Understanding  and  Learning 

So,  what  might  students  get  from  an  educational  experience  like  Archaeotype  that 
they  wouldn't  get  from  a  regular  class,  and  what  might  they  get  from  a  regular  class  that 
they  wouldn't  get  in  Archaeotype?  In  a  regular  class  on  Greek  and  Roman  history,  the 
students  would  probably  learn  more  facts  about  history  (because  they  are  devoting  all  their 
time  to  learning  such  facts)  than  the  Archaeotype  students  would  learn,  but  the  Archaeotype 
students  would  probably  remember  the  facts  they  do  learn  longer  and  have  a  greater 
understanding  of  them  and  historical  reasoning.  ThuL  if  given  an  objective  test  of  memory 
for  Greek  and  Roman  history  facts  at  the  end  of  the  course,  a  standard  class  would  probably 
do  better  than  an  Archaeotype  class,  but  a  year  or  two  later  the  Archaeotype  class  would 
probably  do  better.  More  importantly,  if  we  examined  essays  arguing  for  some  historical 
conclusion,  then  we  would  expect  the  Archaeotype  students  to  be  much  more  sophisticated 
than  the  regular  students  (in  fact,  the  reports  from  current  Archaeotype  students  seem  quite 
sophisticated  in  terms  of  language,  argument  structure,  citations,  etc.)  -  and  thus 
demonstrate  a  much  deeper  understanding  of  historical  facts  and  reasoning.  We  are  in  the 
midst  of  conducting  such  an  investgation  of  content  learning,  but  do  not  have  She  results  to 
report  yet. 

However,  more  than  these  particulars  of  the  topic  area  for  a  class,  an  Archaeotype* 
type  educational  experience  should  teach  students  to  examine  any  situation,  make  relevant 
observations  and  measurements,  organize  these  materials,  search  out  related  bodies  of 
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knowledge,  organize  all  this  information  and  use  it  to  draw  compelling  conclusions  and 
make  useful  recommendations.  Thus,  the  strongest  test  of  student  learning  and 
understanding  from  Archaeotype  would  be  to  compare  their  ability  to  investigate  and  make 
conclusions  and  recommendations  in  an  entirely  different  and  unrelated  situation  to  the 
ability  of  students  who  have  not  had  an  Archaeotype  experience  to  do  the  same.  That  is 
what  we  did  in  the  study  reported  here. 

In  the  study  wo  conducted,  the  students  were  given  a  booklet  describing  four 
psychology  experiments  examining  how  people  remember  lists  of  words.  The  students  had 
to  examine  the  basic  oheservations,  report  on  the  results  of  the  studies,  find  the  patterns, 
devise  explanations  and  argue  for  those  explanations.  They  were  also  given  some 
background  readings  in  the  psychology  of  memory.  The  Dalton  students  who  had  been 
through  the  Archaeotype  program  were  compared  to  students  from  the  Grace  Church  School 
(who  also  had  some  data-analysis  experience  from  going  through  The  Voyage  of  the  Mimi 
program  from  Scholastic  Publishers). 

Method 

Participants 

The  experimental  group  was  20  sixth-grade  students  who  had  participated  in  the 
Archaeotype  program  at  the  Dalton  School,  an  independent  school  located  on  the  east  side  of 
Manhattan.  The  control  group  was  20  sixth-grade  students  who  attended  the  Grace  Church 
School,  an  independent  school  also  located  on  the  east  side  of  Manhattan. 

Materials 

Students  in  the  two  groups  were  given  a  ten-page  document  (the  assignment 
booklet)  divided  into  two  parts.  The  first  part  described  the  results  of  four  memory  studies 
as  follows: 

(1)  in  study  1  subjects  listened  to  21)  words  spoken  at  the  rate  of  one  word 
per  second  and  then  immediately  recalled  them 

(2)  in  study  2  subjects  listened  to  the  same  words  spoken  at  the  rate  of 
one  word  every  three  seconds  and  then  immediately  recalled  them 

in  study  l\  subjects  listened  to  the  same  words  spoken  at  the  rate  of 
one  word  per  second  but  recalled  them  only  after  performing  an 
unrelated  .']()•  second  task 
(4)  in  study  4  subjects  listened  to  a  different  20  words  (many  of  which 
were  semantically  related)  spoken  at  the  rate  of  one  word  per  second 
and  then  immediately  recalled  them. 

The  second  part  of  the  document  provided  background  readings  on  technical  concepts 
such  as  short-term  memory  and  long-term  memory.  Students  were  asked  to  use  these 
readings  to  interpret  the  results  of  the  four  studies  and  to  present  their  interpretations, 
along  with  practical  recommendations  for  improving  memory,  in  a  written  report, 

Procedure 

Administering  the  Materials  and  Collecting  Student  Reports 

The  study  was  conducted  in  two  2-hour  sessions  (for  a  total  of  4  hours)  spread 
over  two  adjacent  days.  On  the  first  day,  the  experimenter  passed  out  the 
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assignment  booklets,  the  students  paired  up,  the  experimenter  read  the  instructions 
on  the  first  page  of  the  assignment  booklet,  then  the  experimenter  ran  a 
demonstration  of  the  kinds  of  memory  studies  described  in  assignment  booklets.  In 
the  demonstration  the  experimenter  read  a  list  of  20  words  then  the  students  wrote 
down  their  recall  of  them  and  the  experimenter  conducted  a  short  discussion  of  what 
the  results  were.  This  demonstration  was  done  so  that  the  students  could  see  what 
the  studies  described  in  the  assignment  booklets  were  like.  After  the  demonstration, 
the  students  proceeded  to  work  on  the  assignment  in  groups  of  two.  While  doing  the 
assignment  the  students  were  free  to  use  any  of  the  resources  in  the  Dalton  and 
Grace  Church  School  buildings  (computers,  libraries,  etc.)  including  asking  the 
experimenter  for  clarification  and  information  questions  (the  same  experimenter 
conducted  all  sessions).  At  the  end  of  the  2-hour  period  on  the  second  day,  the 
students  handed  in  their  reports  and  all  the  work  they  had  done  in  folders.  The 
experimenter  then  lead  a  half-hour  discussion  of  the  study. 

Analysis  of  Student  Reports 

We  devised  a  rubric  for  evaluating  three  dimensions  of  the  student  reports  pattern 
recognition,  argumentation,  and  data  representation.  Given  the  emphasis  on  data 
interpretation  in  the  Archaeotype  program,  we  accorded  the  most  weight  to  the  dimension  of 
argumentation,  as  indicated  by  the  following  distribution  of  points: 

(1)  pattern  recognition  (20  points) 

(2)  argumentation  (30  points) 

(3)  data  representation  (10  points) 

In  principle,  students  could  receive  a  total  of  60  points,  though  we  should  point  out  that  the 
rubric  was  designed  to  reflect  what  might  be  described  as  expert  responses  to  the  task. 
This  emphasis  on  high  standards  is  in  keeping  with  the  larger  movement  in  educational 
reform  that  is  often  referred  to  as  authentic  assessment. 

Pattern  Recognition.  Students  received  1-2  points  for  describing  each  of  the  following 
intra-study  patterns: 

(1)  in  study  1  the  pattern  of  last  words /first  words /middle  words  (with 
middle  words  highly  attentuated) 

(2)  in  study  2  the  pattern  of  last  words/  first  words  /middle  words  (with 
middle  words  more  developed) 

(3)  in  study  3  the  pattern  of  first  words /middle  words /last  words(with  last 
words  highly  attenuated) 

(4)  in  study  4  the  pattern  of  last  words /words  grouped  in  semantic 
categories  (with  last  words  relatively  attenuated) 

In  addition,  students  received  1-2  points  for  describing  each  of  the  following  cross- 
study  patterns  that  relate  to  number  of  words  recalled: 

(5)  more  words  were  recalled  in  study  2  than  in  study  1 

(6)  fewer  words  were  recalled  in  study  3  than  in  studies  1  and  2 

(7)  more  words  were  recalled  in  study  4  than  in  studies  1,2,  and  3 
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In  effect,  the  number  of  words  m  ailed  in  the  studies  can  be  ranked  in  the  following  order: 
study  4   >  .study  2   >  study  1  >  study  3 

Apart  from  these  major  patterns,  students  received  1-6  points  tor  noticing  other 
significant  patterns  (i.e.,  1-2  points  up  to  three  patterns):  for  example,  in  studies  1  and  2 
when  middle  words  were  recalled,  they  often  formed  associative  pairs  (e.tf.,  cup/wuivr)\  or  in 
study  4  the  most  salient  semantic  categories  were  those  involving  fruit  and  animals  as 
opposed  to  those  involving  furniture  and  transportation  (i.e.,  words  in  these  categories  were1 
recalled  not  only  more  frequently  but  earlier  in  the  sequence);  and  within  the  various 
categories,  certain  words  which  function  as  prototypes,  tended  to  be  recalled  first:  lor 
example,  con*  lor  the  category  of  clothing  and  chair  for  the  category  of  furniture. 

Kxplanat |op  arid  Ay^uipentation.  Students  were  expected  to  draw  on  the 
had*  gi omul  readings  to  develop  arguments  supporting  hypotheses  about  the  patterns  tliev 
observed  in  the  four  studies.  As  a  consequence,  arguments  that  drew  appropriately  on  tin* 
background  readings  were  awarded  1-4  points  each,  whereas  arguments,  which  did  not 
draw  on  the  background  readings,  were  awarded  1-2  points  each.  Here  are  local  arguments 
iIimI  loiild  lie  used  in  interpreting  major  patterns  in  the  four  studies: 

( I )  in  study  1  short  term  memory  explains  the  fact  that  the  last  words  urn 
the  lust  recalled 

(lil  in  ntuily  2  increase?  in  time  -  and  thus  deeper  processing  in  long-term 
memory  explains  the  fact  that  more  words  can  be  recalled  (especially, 
lite  middle  words  that  can  be  meaningfully  associated) 

CO  in  study  >\  the  intervening  30-second  task  is  used  to  explain  not  only 
the  fact  the  last  words  are  no  longer  recalled  first  (i.e.,  short-term 
memory  is  no  longer  operating)  but  fewer  total  words  are  recalled  (i.e., 
long  term  memory  is  diminished  as  well) 

( I)  in  atmlv  'I  the  presence  of  semantically  related  words  is  used  to  explain 
the  lint  that  not  only  are  more  words  recalled  but  the  sequence  in 
winch  they  lire  recalled  (i.e.,  semantically  related  words  tended  to  he 
inouped) 

In  addition  to  local  argumentation,  students  were  given  credit  for  global 
ill t'.iiim  itt iiliou  (i<  K  ,  these  four  studies  suggest  that  meaningful  associations  among 
individual  woidn  Hi  the  most  powerful  factor  in  word  recall).  They  were  given  1  V.  point!,  il 
Miib  ni louneiit ution  wiin  presented  without  the  background  readings,  1-4  points  it  it  wait 
pieaenti't)  with  the  background  readings, 

An  tti  the  1 1 ii ii  1  recommendations  in  the  report,  students  were  given  I  4  poind.  lm 
inniiudiiiH  them  in  the  data  (e.g.,  ample  time  should  be  provided  so  that  mciimngtul 
unhtif  ml  nuiM  inn  be  (aimed  between  the  items  to  be  remembered)  and  I  I  pumth  l<>! 
giuiimiiug  them  in  the  background  readings  (e.g.,  meaningful  unsocial loin*  ihoulil  be 
developed  mi  I  In  iS  material  can  be  transferred  from  short-term  memory  to  l<me«  leim 
iiiemoi  v) 

Mtiidciila  weie  also  ^iven  12  points  whenever  they  displnyed  legitimate  lm  in:  nl 
ullei  native  explanation  lor  the  same  phenomena  (for  example,  in  study  lour  the  tin  t  that 
ml  tended  to  occur  early  among  tin*  recalled  words  eould  have  been  explained  by  1 1 n>  fait 
that  it  was  among  the  lust  woiiIh  presented  (i.e.,  short-term  memory)  and/or  the  fact  it 


serves  as  a  prototype  of  the  'animal1  category  (i.e.,  members  of  such  a  category,  as 
ment  ioned,  tend  to  occur  hefore  members  of 'furniture'  or  'transportation1  categories). 


S)ata  Representation.  Students  were  given  credit  if  they  used  numerical  and/or 
graphic  methods  to  represent  major  patterns  in  the  four  studies.  With  respect  to  numerical 
methods,  they  received  1-2  points  if  they  calculated  the  means  for  significant  patterns  such 

as 

(1)  the  total  number  of  words  recalled  in  each  study 

(2)  the  number  of  first  words,  middle  words,  and  last  words  recalled  in 
studies  1-3 

(3)  the  number  of  words  recalled  in  the  semantic  categories  as  well  as  the 
number  of  last  words  recalled  in  study  4. 

Students  received  an  additional  1-2  points  if  they  used  these  means  to  establish 
significant  proportions  such  as 

(1)  the  relative  weighting  of  first  words,  middle  words,  and  last  words  that 
were  recalled  in  studies  1-3 

(2)  the  relative  weighting  of  last  words  and  associated  words  (i.e..  those  in 
the  semantic  categories)  that  were  recalled  in  study  4. 

As  to  graphic  methods  of  representation,  students  were  given  1-6  points  for 
appropriate  use  of  such  methods.  These  methods  include  bar  graphs  that  represent 
the  proportions  of  different  kinds  of  words  recalled  in  the  four  studies.  With  respect 
to  studies  1-3,  the  line  graph  of  proportion  recalled  ploted  against  serial  position 
(usually  called  "the  serial  position  curve")  could  have  been  used  to  represent  the 
major  patterns  constituted  by  fir?t  words/middle  words/last  words.  Alternatively, 
they  could  have  used  a  flow  chart  to  represent  the  input/output  relations  for  short- 
term  and  long-term  memory  in  these  studies.  With  respect  to  study  4,  they  could 
haw  used  tree -structures  to  represent  membership  in  the  major  semantic  categories. 

Results 

We  present  the  results  in  Table  1.  The  numbers  in  this  table  are  the  means  for  the 
Archaeotype  group  and  the  Control  group.  The  total  possible  score  overall  was  60  points, 
although  this  represents  all  that  could  conceivably  be  found,  not  what  any  precollege 
.student  could  attain  -  only  a  specialist  in  the  psychology  of  memory  would  have  a  chance  of 
getting  all  these  points.  Thus,  the  important  aspect  of  these  numbers  is  not  their  absolute 
value,  hut  how  the  Archaeotype  and  Control  groups  compare.  This  comparison  is  striking:  in 
total  (the  first  column  in  Table  1),  the  Archaeotype  group  scored  31%  higher  than  the  Control 
group  (25.2  vs  19.2    out  of  a  possible  HO),  and  this  difference  was  very  statistically 
significant,  /(38)=2.22,  p<.02.  To  do  this  statistical  analysis  ind  the  others  reported  later, 
we  assigned  each  student  the  score  of  the  report  created  by  the  group  (here,  each  group  is  a 
pair)  that  they  were  in,  then  calculated  a  t  test  to  see  how  big  the  difference  between  the 
means  of  the  Archaeotype  student  scores  and  the  Control  student  scores  were  compared  to 
the  variance  of  these  scores  within  the  Archaeotype  group  and  within  the  Control  group. 
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Table  1 

Quantitative  Analysis  of  Reports  Written  by  Students 
in  the  Archaeotype  Group  and  the  Control  Group 


Total 


Pattern  Explanation  and 
Recognition  Argumentation 


Data 


Representation 


Archaeotype  Group  25.2 


10.6 


13.8 


0.8 


Control  Group 


19.2 


9.6 


8.0 


1.6 


As  described  earlier,  this  overall  total  score  breaks  down  into  subscores  for 
recognizing  the  patterns  in  the  observations  (Pattern  Recognition),  explaining  the  patterns 
and  arguing  for  those  explanations  (Explanation  and  Argumentation),  and  converting  the 
observations  into  forms  that  could  provide  insight  (Data  Representation).  This  breakdown 
shows  that  the  overall  Archaeotype  superiority  was  almost  totally  caused  by  a  73%  higher 
performance  for  the  Archaeotype  students  in  the  important  Explanation  and  Argumentation 
area  (13.8  vs  8.0  out  of  a  possible  30  points).  Statistically  also,  this  is  a  highly  significant 
difference,  f(38)=3.34,  p<.001.  There  was  also  a  slight  difference  in  favor  of  the  Archaeotype 
students  in  the  Pattern  Recognition  srores  (10.fi  vs  9.6  out  of  a  possible  20),  but  that 
difference  was  not  even  close  to  being  statistically  significant  so  we  have  to  discount  it, 
*(38)=0.76,  p>.2. 

The  Data  Representation  scores  held  two  surprises  for  us.  The  first  surprise  is  that 
they  were  so  low  (lfi%  mid  H%  of  the  possible,  compared  to  27%-53%  of  the  possible  in  the 
other  areas):  neither  the  An  havotypv  students  nor  the  Control  students  used  means, 
proportions,  graphics  nor  diagrams  in  their  discussions  -  they  merely  talked  about  one 
condition  described  in  tin*  experimental  materials  being  greater  than  another.  The  second 
surprise  is  that  the  Control  students  scored  better  than  the  Archaeotype  students  (1.6  vs  0.8 
out  of  a  possible  10)  to  u  significant  degree,  /(3H)=1.95,  p<.05.  However,  the  Control 
advantage  was  totally  due  to  these  students  putting  the  observations  into  a  database 
program  on  the  computers  (part  of  Murtmoft  Workn,  which  they  were  accustomed  to  using) 
and  calculating  means.  For  example,  one  pair  of  students  in  the  control  group  displayed  the 
database  shown  in  Appendix  (',  Pigure  h  This  use  of  databases  was  a  potentially 
valuable  move,  but  the  control  students  did  nut  exploit  this  analysis  for  Pattern  Recognition 
and  Explanation- Argumentation.  The  An  huvutypv  students  did  not  show  comparable  use  of 
database  or  spreadsheet  programs  and  thus  scored  lower  on  Data  Representation.  Taken 
together  these  results  show  that  the  students  both  need  to  have  experience  using  computer 
programs  for  manipulating  data,  but  they  also  need  practice  using  them  meaningfully  as 
part  of  their  work  in  analyzing  authentic  tasks. 


The  results  showed  an  impressive  ability  on  the  part  of  the  Archaeotype  students  to 
create  explanations  of  observations  and  argue  for  the  validity  of  those  explanations  using  a 
mixture  of  their  own  terms  and  ideas,  and  the  technical  terminology  and  concepts  provided 
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by  background  readings  in  a  research  literature.  They  also  did  well  in  recognizing  patterns 
in  the  observations,  but  not  significantly  better  than  the  control  group  we  compared  them  to. 
In  fact,  the  similar  performance  of  the  Dalton  School  Archaeotype  students  and  the  Grace 
Church  School  Control  students  on  the  Pattern  Recognition  portion  of  the  assignment 
provides  assurance  that  the  two  groups  were  comparable,  which  makes  the  much  higher 
performance  of  the  Archaeotype  students  on  Explanation -Argumentation  all  the  more 
impressive.  However,  we  need  to  also  recognize  that  the  basic  patterns  in  the  observations 
the  students  were  analyzing  were  fairly  easy  to  see  -  particularly,  after  the  demonstration 
and  discussion  conducted  by  the  experimenter  in  the  beginning  of  the  sessions.  It  may  be 
that  if  the  patterns  being  searched  for  had  been  less  apparent  then  there  would  have  been 
more  of  a  difference  in  Pattern  Recognition  between  the  Archaeotype  students  and  the 
Control  students.  In  fact,  a  study  we  have  done  comparing  performance  on  another  program 
with  a  similar  design  (Galileo  which  teaches  science  to  high  school  students  through 
astronomy)  found  pattern -recognition  differences  when  the  patterns  were  much  harder  to 
see. 

The  Archaeotype  students  actually  did  worse  than  the  Control  students  in  Data 
Representation,  although  both  groups  scored  rather  low  in  this  area.  It  is  disappointing 
that  the  Archaeotype  students  did  not  use  even  such  rudimentary  ways  of  representing  data 
as  counts,  means  and  proportions.  At  least  some  students  in  the  Control  group  managed  to 
do  some  counting  and  means  through  entering  the  observations  into  a  computer  database 
program  they  were  accustomed  to  using.  Ideally,  the  students  would  even  have  used 
visualization  techniques  like  graphs  and  diagrams  to  reveal  patterns  in  the  observations 
and  to  argue  for  their  explanations.  Archaeotype  would  seem  a  natural  context  within 
which  to  introduce  the  powerful  idea  of  representing  information  in  different  forms  to  gain 
insight. 

References 

Bednar,  A.K.,  Cunningham,  D.,  Duffy,  T.M.,  and  Perry,  J.D.  (1991).  Theory  into 

practice:  How  do  we  link?  In  G.J.  Anglin  (Ed.)  Instructional  technology:  Past, 
present  and  future.  Englewood,  CO:  Libraries  Unliminted,  Inc. 

Brown,  J.S.,  Collins,  A.  and  Duguid,  P.  (1989).  Situated  cognition  and  the  culture  of 
learning.  Educational  Researcher,  19,  32-42. 

Cognition  and  Technology  Group  at  Vanderbilt  (1990).  Anchored  instruction  and  its 
relation  to  situated  cognition.  Eductional  Researcher,  20,  2-10. 

Collins,  A.,  Brown,  J.S.,  and  Newman,  S.E.  (1990).  Cognitive  apprenticeship.  In  L.B. 
Resnick  (Ed.).  Knowing,  learning  and  instruction.  HillHdalc,NJ:  Erlbaum. 

Jonussen,  D.H.  (1991)  Objectivism  versus  Constructivism:  Do  wc  need  a  new 

philosophical  paradigm?  Educational  Technology  Research  and  Development, 
39,  5-14. 

McClintock,  R.  (1971) .  Toward  a  place  for  study  in  a  world  of  instruction.  Teachers 
College  Record,  72,  405-416. 

Palmer,  R.E.  (1969)  Hermeneutics.  Evanston,IL:  Northwestern  University  Press. 
Spiro,  R.J.,  Feltovich,  P.J.,  Jacobson,  M.J.  and  Coulson,  R.L.  (1991).  Cognitive 

flexibiltiy,  constructivism,  and  hypertext.  Educational  Technology,  21,  24-33, 


61 


73 


Title: 

Constructivist  Design  of  Graphic  Computer  Simulations 


Authors: 

John  B.  Black 
Will  Thalheimer 
Hilary  Wilder 
Donna  de  Soto 
Paul  Picard 

Teachers  College 
Columbia  University 


63 

74 


Several  proposals  have  appeared  recently  arguing  that  the  design  of  educational 
systems  should  take  a  constructivist  rather  than  an  instructional  approach  (e.g.,  Bednar, 
Cunningham,  Duffy,  and  Perry,  1991;  Harel  and  Papert,  1991;  Newman,  Griffin  and  Cole, 
1989).   However,  while  there  are  established,  integrative  frameworks  for  guiding  the  design 
of  instructional  systems  (e.g.,  Gagne,  Briggs  and  Wager,  1992;  Reigeluth,  1983; 
Romiszowski,  1981),  there  does  not  appear  to  be  an  integrated  framework  for  guiding  the 
design  of  constructivist  educational  systems.  We  take  a  first  step  in  this  direction  by 
proposing  six  interrelated  principles  for  constructivist  design.  Versions  of  the  individual 
princples  have  appeared  in  a  variety  of  places,  but  we  have  tried  to  formulate  them  in  such 
a  way  that  they  work  together. 

The  six  principles  of  constructive  design  that  we  propose  and  their  sources  are: 

1.  Set  the  stage  but  have  the  students  generate  the  knowledge  for 

themselves  as  much  as  possible  (Jacoby,  1978;  Black,  Carroll 
and  McGuigan,  1987) 

2.  Anchor  the  knowledge  in  authentic  situations  and  activities 

(Cognition  and  Technology  Group  at  Vanderbilt,  1990) 

3.  Use  the  cognitive  apprenticeship  methods  of  modeling,  scaffolding, 

fading  and  coaching  to  convey  how  to  construct  knowledge 
in  authentic  situations  and  activites  (Collins,  Brown  and 
Newman,  1990) 

4.  Situate  knowledge  in  multiple  contexts  to  prepare  for  appropriate 

transfer  to  new  contexts  (Gick  and  Holyoak,  1983) 

5.  Create  cognitive  flexibility  by  ensuring  that  all  knowledge  is  seen 

from  multiple  perspectives  (Spiro,  Feltovich,  Jacobson  and 
Coulson,  1991) 

6.  Have  the  students  collaborate  in  knowledge  construction  (Johnson 

and  Johnson,  1975) 

We  have  created  two  graphic  computer  simulations  for  teaching  high  and  middle 
school  students  about  how  business  organizations  and  financial  systems  work  (these 
programs  are  currently  being  used  in  a  number  of  New  York  City  public  high  schools).  The 
two  simulations  are  Parkside,  which  simulates  managing  a  hotel,  and  Guestwcar,  which 
simulates  managing  a  clothing  manufacturer  (formative  work  showed  these  topics  to  be 
motivating  to  the  urban  student  population  we  targeted).  Both  of  these  simulations  are 
implemented  in  the  Toolbook  hypermedia  authoring  environment  running  in  Windows  (3.0 
or  higher)  on  386/486-based  IBM  and  equivalent  computers.  Here,  we  use  the  design  and 
use  of  these  simulations  to  illustrate  how  the  constructive  design  principles  can  be  applied. 

For  example,  in  Parkside  we  set  the  stage  for  student  knowledge  generation 
(Principle  1)  by  providing  an  authentic  hotel  environment  for  the  students  to  interact  with, 
while  providing  supplemenatry  information  that  the  students  can  use  to  figure  out  how  to 
accomplish  goals  and  deal  with  problems  that  arise  in  this  setting.  As  they  nit  in  their 
simulated  office  or  wander  around  the  simulated  hotel,  various  problems  arise  (through 
interacting  with  simulated  people,  reading  memos,  answering  phone  calls,  etc.)  that  they 
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have  to  deal  with.  They  Rather  relevant  information  that  they  can  use  to  figure  out  how  to 
deal  with  the  problems  through  a  variety  of  information  sources  that  the  simulation  provides 
(e.g.,  an  icon-activated  "Managers  Guide"  that  whenever  requested  can  provide  context- 
sensitive  advice  and  feedback;  an  icon-activated  quantitative  report  that  provide  a  variety  of 
performance  indices  like  occupancey  rates,  income,  customer  satisfaction,  newspaper 
articles,  advice  from  staff  at  simulated  meetings  that  can  be  called  at  any  time,  etc.).  This 
fairly-realistic  graphic  display  and  the  underlying  functional  relations  (implemented  as  a  set 
of  complex  if-then  rules  and  equations)  provides  the  authentic  situations  and  activites  that 
anchor  the  material  covered  (Principle  2).  Thus  the  students  deal  with  business  concepts 
like  supply  and  demand  or  the  importance  of  a  trained  workforce  not  in  the  abstract,  but  in 
terms  of  concrete  actions  needed  to  make  their  hotel  work  better. 

The  students  get  started  on  the  Parkside  simulation  by  having  the  teacher  "walk- 
through" an  initial  portion  of  the  simulation  designed  for  this  purpose  (module  0)  verbalizing 
how  he  or  she  is  thinking  through  the  situations  and  problems  encountered.  Gradually  the 
teacher  fades  out  their  contribution  to  this  "walk-through"  and  the  thinking  is  taken  over  by 
the  students  in  a  class  discussion.  Then  the  students  proceed  through  the  rest  of  the 
simulation  (modules  1-5)  working  in  groups  around  computer  workstations  while  the  teacher 
walks  around  the  room  providing  help  (scaffolding)  and  advice  (coaching).  Thus  the  students 
an*  introduced  to  and  guided  through  the  simulation  using  a  cognitive  apprenticeship 
approach  (Principle  3).  Cognitve  apprenticeship  is  also  provided  in  the  simulation 
program  itself  by  the  "Managers  Guide"  and  other  information  aides  mentioned  earlier  that 
the  students  can  activate  whenever  needed. 

The  students  work  through  the  simulation  in  groups  (generally  four  per  group  in  the 
current  sites)  around  each  computer  so  that  they  can  collaborate  on  exploring  the  simulated 
world  and  making  decisions  (Principle  6).  Since  the  various  group  members  each  have 
their  own  perspectives  on  the  material,  the  discussion  in  the  groups  and  the  later  discussion 
between  the  groups  provide  multiple  perspectives  (Principle  5).  Multiple  perspectives  are 
also  provided  by  design  in  the  comments  and  suggestions  made  by  the  simulated  people  of 
the  hotel  world.   Most  of  the  knowledge  that  we  want  the  students  to  learn  appears  in  more 
than  one  context  in  the  Parkside  simulation,  and  later  is  covered  again  in  the  Guestivear 
simulation,  which  provides  them  in  the  very  different  context  of  a  simulated  clothing 
manufacturer  (Principle  4). 

Problems  with  Initial  Field  Test 

For  an  initial  test  of  the  effects  of  the  Parkside  and  Guestwear  simulation  programs, 
we  placed  them  in  two  New  York  City  public  high  schools  using  computer  laboratories 
donated  for  this  purpose  by  IBM.  These  schools  were  quite  happy  to  accept  this  computer 
equipment  and  have  us  run  our  simulations  there  as  part  of  their  Introduction  to 
Occupations  course,  and  the  students  received  the  programs  enthusiastically  (e.g., 
attendance  skyrocketed  to  virtually  100%),  but  the  school  adminstrations  balked  at  moving 
beyond  this  superficial  involvement. 

The  first  sign  of  trouble  was  the  school  administrations  refusal  to  give  the  teachers 
release  time  to  receive  training  in  how  to  use  the  program  and  the  teachers  were  unwilling  to 
devote  their  own  time  to  this.  In  fact,  the  depth  of  teacher  training  needed  30on  became 
apparent:  the  teachers  not  only  had  no  experience  using  simulations  (and  not  much  using 
computers  in  general),  as  part  of  their  teaching  but,  the  mode  of  thought  embodied  in  the 
simulation  was  completely  alien  to  them.  In  particular,  the  simulations  were  designed  to 
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inculcate  a  mental  model  style  of  thinking  (e.g,  Gentner  and  Stevens,  1983;  Mandinach, 
HW9)  about  business  and  financial  systems.  Thus  we  wanted  the  students  to  learn  that 
making  decisions  can  affect  a  system  of  interacting  entities  in  various  ways  (e.g.,  increasing 
hotel  room  rates  might  increase  initial  revenue  but  depending  on  circumstances  might 
decrease  occupancy  rates  and  change  the  image  of  the  hotel,  and  so  forth).  The  teachers 
never  did  grasp  this  style  of  thinking:  they  kept  insisting  that  there  was  no  way  they  could 
teach  using  the  simulation  unless  we  told  them  what  the  "right"  answer  is  at  each  decision 
point.  In  the  end,  our  project  team  had  to  take  over  the  parts  of  the  class  using  the 
simulations. 

Even  worse,  when  the  time  came  for  us  to  administer  a  test  (presenting  a  new 
business  case  and  having  the  students  make  decisions  and  explain  their  decisions),  the 
Principals  of  the  schools  refuse  to  let  us  test  their  students.  It  seems  at  the  point  they  had 
gotten  what  they  wanted  from  the  project  (e.g.,  new  computer  labs)  and  did  not  want  to 
chance  our  revealing  any  weaknesses  in  their  students.  Thus,  we  were  not  able  to  evaluate 
the  effects  of  the  simulations,  but  we  were  left  with  an  appreciation  of  the  difficulties  of 
accomplishing-  anything  within  the  current  organizational  structure  of  the  New  York  City 
Public  Schools. 

A  Quick-and-Dirty  Field  Test 

Fortunately,  we  were  able  to  use  an  Applied  Economics  class  in  a  Connecticut  public 
hif'h  school  to  do  a  pilot  pre-and  post-test  using  our  assessment  instrument  with  a  class 
that  used  the  Parkside  simulation.  The  teacher  of  this  class  is  a  graduate  student  in  our 
department  and  thus  has  the  requisite  understanding  of  computers,  simulations  and 
cognition  to  make  meaningful  use  of  the  simulation.  The  tests  administered  described  new 
business  cases  (e.g.,  managing  a  pizza  parlor)  and  required  the  students  to  make  a  series  of 
decisions  about  the  business  (choose  from  a  set  of  alternatives),  acquire  new  vocabulary 
about  business  and  economics,  and  explain  their  reasoning  in  essay  questions,  the  class 
was  composed  of  16  students,  and  12  of  them  completed  both  the  pre-test  and  the  post-test 
(which  were  equivalent  but  with  different  content).  These  12  students  improved  form  the 
pre-test  to  the  post-test  10%  on  the  multiple-choice  decisions,  13%  on  the  vocabulary  (being 
able  to  explain  what  the  terms  mean)  and  22%  on  the  quality  of  reasoning  expressed  in 
their  essay  answers.  Thus,  while  far  from  definitive,  these  results  suggest  that  the  students 
are  indeed  learning  from  the  simulations  and  improving  most  in  higher-level  thinking  (the 
essays).  Particularly  interesting  was  the  teachers  report  that  2  of  the  12  students  had  been 
uninvolved  in  the  classroom  part  of  the  course,  but  became  enthusiastic  leaders  during  the 
simulations.  This  suggests  that  simulations  like  Parkside  may  reach  students  who  do  not 
relate  to  other  methods  of  instruction. 
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Using  an  adult  motivational  msttuttional  design  model,  the  authors 
demonstrate  the  application  of  prescnbvd  stnitcuiCH  into  instructional  software 
designed  to  teach  basic  statistics  concepts.  The  Ihmv  software  is  compared  to 
the  enhanced  motivational  vcrsmn.  Important  motivational  strategies  are 
explained  and  implications  and  applications  of  motivating  design  of 
instruction  for  adults  are  discussed. 
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Introduction 


Many  researchers  (Aslanian  &  Brickel,  1980;  Cross,  1981;  Galbraith,  1990  & 
1991;  Houle,  1961;  Knowles,  1980;  Wlodkowski,  1985;  Zemke  &  Zemke,  1981)  have 
proposed  that  adults  have  specific  and  rather  unique  motivational  needs  in  instructional 
settings.  These  needs  may  be  due  to  any  number  of  changes  that  take  place  over  time:  (a) 
life  experiences  (Knowles,  1980),  (b)  transitions  in  life  (Cross,  1981),  or  (c) 
learned  attitudes  over  time  as  a  major  factor  in  adult  learning  (Wlodkowski,  1985).  As  a 
result,  it  would  seem  that  motivating  instruction  for  adult  learners  should  be  designed 
with  special  considerations  in  mind. 

Motivational  Instructional  Design 

Instructional  motivation  increases  the  learners1  effort  and  attracts  learners  to 
the  instructional  content  and  methods  (Keller,  1983).  This  means  that  instructional 
motivation  has  two  components,  it  is  appealing  and  effort  generating.  Motivational 
instruction,  therefore,  facilitates  an  appeal  or  interest  within  the  learner  and  it  also 
stimulates  an  increase  in  the  effort  or  time  on  task  of  the  learner.  This  dual 
characteristic  of  instructional  motivation  has  been  supported  in  previous  studies  (Bohlin, 
Milheim,  &  Viechnicki,  1990a  &  b). 

Keller  (1987),  Keller  and  Suzuki  (1988),  and  Keller  and  Kopp  (1987)  specifically 
identify  four  categories  of  motivating  strategies  in  learning  situations:  attention, 
relevance,  confidence,  and  satisfaction  (ARCS).  The  ARCS  Model  proposes  that  motivating 
instruction  contains  strategies  from  some  or  all  of  these  four  categories.  The 
instructor/designer,  therefore,  can  promote  motivation  by  using  appropriate  strategies. 

The  attention  of  the  learner  can  be  promoted  through  procedures  that  increase 
curiosity,  interest,  or  arousal  by  using  humor,  variety,  enthusiasm,  etc.  The  relevance  of  the 
instruction  refers  to  the  personal  needs  of  the  learner  and  can  be  facilitated  by  matching  the 
instruction  to  learners'  goals,  making  the  benefits  clear,  keeping  the  challenge  level 
appropriate,  etc.  The  confidence  of  the  learner,  defined  as  the  learner's  expectancy  for 
success,  can  he  increased  by  strategies  such  as  clearly  indicating  the  requirements  for 
success,  providing  a  low  risk  environment,  and  giving  accurate  attributional  feedback. 
The  learners'  satisfaction  depends  upon  the  extent  to  which  the  rewards  gained  are 
pcrcc ivori  as  appropriate  and  consistent  with  their  expectations,  and  can  be  promoted  by 
providing  appropriate  recognition  for  success,  giving  informative  and  corrective 
feedback,  etc. 

This  model  was  developed  through  inductive  and  deductive  methods  focused  on  K- 
1 2  classrooms.  But  if  theorists  are  correct,  these  data  may  not  generalize  to  adults.  In 
order  to  better  match  motivating  strategies  to  adult  instruction,  we  must  go  beyond  the 
ARCS  look  at  the  needs  of  adults. 

Previous  studies  (Bohlin,  Milheim,  &  Viechnicki,  1993a  &  b)  have  identified 
specific  strategies  from  the  ARCS  categories  and  from  adult  learning  theory  that  are 
especially  motivating  for  adults.  These  strategies  have  been  integrated  into  an  adult 
instructional  motivation  design  model,  which  has  been  referred  to  as  the  "Golden  ARCS 
Model."  The  strategies  from  this  model  should  provide  a  better  match  when  designing 
motivating  instruction  or  materials  for  adult  learners.  Many  of  these  strategies  can  be 
easily  integrated  into  instruction  and  materials  in  ways  that  will  increase  the  interest 
and  effort  of  older  learners. 
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Integrating  Strategies  into  Instruction 


Based  on  the  above  research,  the  authors  have  developed  a  number  of  suggestions 
for  incorporating  motivational  strategies  into  instructional  materials.  While  the  majority  of 
the  strategies  listed  below  are  focused  primarily  on  their  use  within  computer-based 
instruction,  many  could  also  be  easily  adapted  for  use  with  other  forms  of  instructional 
media. 

Attentipn 

In  terms  of  attention,  specific  computer-based  materials  could  begin  with 
several  facts  that  contradict  the  learner's  previous  perceptions  concerning  the  material 
included  in  the  program  (Keller,  1987).  In  addition  to  reducing  the  overall  anxiety  of 
the  student,  this  strategy  may  gain  the  attention  of  the  learner  since  the  material  is 
somewhat  unexpected  based  on  previous  experience.  Similar  unexpected  events  (within 
reason)  can  also  be  integrated  throughout  other  parts  of  the  program  to  help  engage  the 
learner  in  a  deeper  level  of  interest  and  attention  (Keller  &  Suzuki,  1988). 

Content -related  anecdotes,  case  studies,  and  biographies  are  also  appropriate 
(Keller,  1987),  since  they  provide  variety  within  an  instructional  program  and  help 
maintain  a  level  of  interest  in  all  of  the  included  material.  Attention  based  on  specific 
instructional  screens  can  be  facilitated  through  the  use  of  traditional  strategies  such  as 
animation,  inverse  text,  or  the  use  of  sound  (Keller  &  Suzuki,  1988;  Rieber  &  Hannafin, 
1988),  although  the  overuse  of  these  strategies  can  interfere  with  the  learning  process 
(Rieber,  1990).  The  integration  of  problem-solving  activities  within  the  lesson  (Keller,  1987) 
also  assists  in  breaking  up  the  potential  monotony  that  can  occur  within  instruction 
delivered  by  computer. 

Figure  1  shows  one  method  for  increasing  attention  within  computer-based 
training,  where  animation  (a  flashing  arrow  in  the  computer  program)  indicates  an 
important  point  on  the  screen.  While  this  type  of  attention-getting  device  should 
obviously  be  only  one  part  of  an  overall  motivational  strategy,  it  can  be  quite  effective  for 
directing  a  learner's  attention  to  a  specific  aspect  of  an  instructional  program. 


Insert  Figure  1  about  here 


Relevance 

For  this  component  in  the  model,  instructional  designers  should  focus  on  showing 
learner  how  the  instructional  materials  relate  to  specific,  current  needs.  Keller  (1987),  for 
example,  suggests  showing  how  specific  instruction  relates  to  future  learner  activities 
in  addition  to  building  on  current  or  existing  skills.  Demonstrating  significant  enthusiasm 
for  the  content  within  the  program  can  also  indicate  to  the  learner  that  the  material  is 
important  and  therefore  relevant  to  their  potential  future  needs. 

More  specific  suggestions  for  increasing  the  relevance  of  an  instructional  module 

include: 

•  the  use  of  personal  pronouns  as  well  as  the  student's  actual  name  to 
provide  a  friendly  and  familiar  atmosphere  (Keller  &  Suzuki,  1988), 

•  the  consistent  use  of  examples  from  a  student's  everyday  job  or  general 
environment,  and 

•  a  description  of  the  application  of  the  content  to  the  real  life  of  the 
learner  where  it  can  be  transferred  after  the  program  is  over. 
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Figure  2  indicates  the  opening  screen  of  a  computer  program  where  the  learner 
is  asked  for  their  social  security  number  as  well  as  his  or  her  first  name.  While  the  number 
is  used  for  tracking  purposes,  the  name  can  be  used  throughout  the  program  for 
personalized  feedback  and  reinforcement  as  well  as  to  increase  the  friendliness  of  the 
experience. 


Insert  Figure  2  about  here 


Confidence 

In  terms  of  confidence,  learners  need  to  be  consistently  reminded  of  their  own 
intrinsic  abilities  and  their  overall  potential  for  success  within  a  program.  In  this  respect, 
the  content  within  an  instructional  module  should  be  organized  on  an  increasing  level  of 
difficulty,  with  the  student  also  being  shown  that  the  material  can  be  learned  through 
personal  effort  even  though  the  content  may  seem  somewhat  complex.  Moderately 
difficult  self-evaluation  tools  can  also  be  provided  to  the  involved  learners  (Keller,  1987)  so 
they  can  assess  their  own  competence  as  they  move  through  the  overall  program. 

With  respect  to  navigation  within  the  program,  the  learner  should  be  permitted  as 
much  control  as  possible,  including  at  least  the  ability  to  page  forwards  and  backward 
from  screen  to  screen  and  to  escape  to  a  menu  at  any  time  (Keller  &  Suzuki,  1988).  This 
type  of  control  will  provide  users  with  the  ability  to  progress  at  their  own  pace  (with 
limited  intervention  from  the  instructional  program)  and  hopefully  increase  their  confidence 
in  their  ability  to  use  the  program  and  learn  the  appropriate  content. 

Several  of  these  suggestions  are  illustrated  in  Figure  3  which  shows  a  simple 
problem  which  can  be  easily  mastered  by  the  learner  based  on  the  skills  and  concepts 
previously  mastered.  This  screen  also  shows  the  availability  of  'Next'  and  'Previous'  choices 
as  well  as  a  button  that  can  return  the  user  to  the  Main  Menu  at  any  point  during  the 
instructional  program. 


Insert  Figure  3  about  here 


Satisfaction 

Finally,  satisfaction  can  be  enhanced  by  a  number  of  different  strategies.  Keller 
(1987),  for  example,  suggests  providing  praise  for  successful  progress  through  an 
instructional  program  as  well  as  verbal  reinforcement  for  a  student's  intrinsic  pride  in 
their  own  task  accomplishment.  He  suggests  varying  the  schedule  of  this  reinforcement, 
with  frequent  praise  provided  when  a  student  is  learning  a  new  task  and  intermittent 
reinforcement  as  the  student  becomes  more  competent  with  the  material. 

In  addition,  Keller  an^  Kopp  (1987)  suggest  allowing  learners  to  use  their  newly 
acquired  knowledge  or  skills  in  a  real  or  simulated  setting  following  their  instructional 
experience,  since  the  successful  utilization  of  these  skills  would  increase  the  student's 
satisfaction  with  the  learning  process  and  th*?ir  potential  ability  to  use  their  newly  acquired 
knowledge.  This  type  of  utilization  could  also  increase  the  confidence  level  of  these 
students  (see  above)  since  they  would  be  more  comfortable  in  their  potential  use  of  these 
new  skills. 
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Discussion 

Adult  instructional  motivation  is  becoming  increasingly  important  as  more  adult 
learners  are  returning  to  instructional  settings,  through  schooling,  training,  and  retraining. 
The  vary  nature  of  students  in  higher  education  is  changing  as  nontraditional  students 
are  returning  to  colleges  and  universities.  Also,  as  societal  changes  accelerate,  progressively 
more  adults  are  forced  to  learn  new  skills  and  processes.  There  is  a  great  need  of 
motivating  materials  for  this  population. 

The  instructional  materials  highlighted  in  this  paper  were  developed  by  integrating 
strategies  and  components  from  a  model  for  the  design  of  motivating  instruction  for 
adults.  It  is  expected  that  these  can  compensate  for  a  lack  of  interest,  relevance,  or 
confidence  of  adult  learners.  This  is  especially  important  for  such  topics  and  statistics, 
which  for  many  carries  with  it  feelings  of  anxiety  and  low  self- efficacy. 

These  kinds  of  materials  can  also  be  used  conduct  research  of  such  questions  as: 

•  What  is  the  minimum  amount  of  motivational  strategies  to  have  achievement 
effects? 

•  What  is  the  minimum  amount  of  motivational  strategies  to  have  affective 
effects? 

•  At  what  point  does  the  use  of  excessive  numbers  of  strategies  become 
demotivating? 

•  What  combination  of  strategies  are  best  for  adults  from  underrepresented 
populations? 

Whether  the  focus  is  on  development  or  research,  more  applications  and 
investigations  must  take  place  in  order  to  assess  the  real  needs  of  adults  in  various 
settings.  In  addition,  more  ways  to  meet  those  needs  must  he  examined. 
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The  graph  below  graphically  depicts  the  data  shown  on  the 
previous  screen.  As  can  be  seen,  there  are  exactly  four 
scores  above  and  four  scores  below  the  median  value  of  25. 
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One  of  the  easiest  ways  to  calculate  the  median  is  to  count 
from  each  end  of  the  distribution  of  scores,  and  then  stop 
when  you  reach  the  middle  score,  which  is  the  median. 
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Figure  1.  Animation  as  an  attention  strategy 
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An  Introduction  to  Statistics 


Type  your  Social  Security  Number, 
Then  Press  the  Return  Key 


123-45-6789 


What  is  your  First  Name?  Michael 


Thank  You,  Michael 
Now,  Click  the  'Next1  Button  to  Continue 
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Figure  2.  Personalization  as  a  relevance  strategy 


The  range  of  a  distribution  of  scores  is  simply  the 
difference  between  the  highest  scores  and  the  lowest. 
This  provides  a  very  simple  measure  of  the  variability  in 
one  group  of  scores  as  compared  to  another  group. 

In  the  following  group  of  scores,  what  do  you  think  the 
range  would  be? 

26,  27,  24,  25,  28,  26,  26,  27 
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Figure  3.  Learner  control  options^as  a  confidence  strategy 
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Professional  Education  and  Case-Based  Instruction 

Educators  in  various  disciplines  such  as  medicine  (e.g.,  Association  of  American 
Medical  Colleges,  1984),  teacher  education  (e.g.  Shulman,  1986),  business  (e.g.,  Mason, 
1992)  and  engineering  (e.g.,  Albright  &  Albright,  1981)  have  expressed  increased 
dissatisfaction  with  the  quality  of  professional  preparation  students  receive  at  universities. 
Key  concerns  are  students'  lack  of  problem  solving  and  critical  thinking  skills.  Content 
coverage  often  is  the  primary  criterion  for  success  in  professional  education  programs 
(Kennedy,  1990),  but  students  are  criticized  for  being  unable  to  transfer  content  knowledge 
to  real  situations  (Kennedy,  1990;  Feltovich,  Spiro,  &  Coulson,  1989). 

In  an  effort  to  give  students  more  experience  with  real  situations,  many  professional 
schools  use  case  studies  in  their  courses.  Originating  in  law  schools  during  the  late  19th 
century,  case-based  instruction  has  been  adapted  in  many  disciplines,  including  business, 
medicine,  and  teacher  education.  Educators  in  these  disciplines  began  using  cases  to  give 
their  students  a  more  field-based,  realistic  education.  Frustrated  by  the  abstractness  of 
lectures  and  the  inconsistencies  of  apprenticeship  (let  alone  the  logistical  difficulties), 
educators  turned  to  actual  court  rulings,  business  interviews,  patient  transcripts,  etc.  to 
help  bring  into  the  classroom  the  environments  for  which  they  were  training  their  students. 
In  case-based  instruction,  students  rend  case  studies  taken  from  actual  events,  analyze 
those  events,  hypothesize  about  problems  in  the  case,  and  recommend  alternative  solutions. 
Students  engage  in  a  dialogue  with  their  professors  about  cases  rather  than  merely  taking 
notes  about  what  professors  espouse  in  lectures. 

There  are  several  benefits  ofte.i  associated  with  learning  via  cases.  Cases  put 
information  in  a  realistic  context  for  sti  dents.  By  contextualizing  information,  students  can 
examine  and  compare  situations  and  come  to  a  better  understanding  of  the  information. 
Because  cases  represent  real  events,  the  concepts  and  skills  learned  in  analyzing  cases  are 
more  easily  transferable  to  'he  real  world  than  those  learned  abstractly.  Cases  also  tend  to 
promote  higher  levels  of  leaining  such  as  analysis,  evaluation  and  problem  solving. 
Furthermore,  case-based  instruction  can  be  helpful  in  bridging  the  gap  between  theory  and 
practice,  promoting  higher-level  objectives,  and  motivating  students  (e.g.,  Christensen,  1987; 
Harrows,  1985). 

Despite  the  oft-cited  benefits  of  case-based  instruction  and  its  widespread  use, 
criticisms  of  professional  education  still  abound.  Some  educators  have  become  frustrated 
with  traditional  cases  because  they  typically  simplify  situations,  lack  richness  of  detail,  and 
are  written  to  illustrate  one  particular  point.  While  cases  seem  to  do  a  good  job  of  putting 
information  in  realistic  contexts,  they  could  be  doing  more  to  help  students  gain  better,  more 
integrated  knowledge  and  skills  and  to  help  them  transfer  these  to  the  working  world,  A 
possible  solution  is  to  create  rich,  complex  cases  —  rase  experiences  that  immerse  students  in 
complex  problems  rather  than  case  studies  that  merely  require  armchair  analysis. 


This  paper  will  discuss  the  need  for  complex  cases  in  higher  education,  particularly 
business  education.  Issues  about  designing,  implementing,  and  learning  from  complex  cases 
will  be  examined  and  an  example  of  a  complex  case  will  be  described. 

The  Case  for  Complex  Cases 

Many  recent  influences  in  educational  psychology  (e.g.,  Brown,  Collins,  &  Duguid, 
1989;  Lave,  1988;  Spiro,  Coulson,  Feltovich,  &  Anderson,  1988)  call  for  the  design  of 
learning  environments  that  are  much  more  contextualized  (i.e.,  situated),  authentic,  and 
apprenticeship-like.  Case-based  instruction  is  usually  considered  a  move  in  this  direction. 
Cases  are  often  recommended  as  an  instructional  method  that  preserves  the  cognitive 
authenticity  of  learning  tasks  without  the  necessity  of  actually  being  in  the  field  (e.g., 
Shulman,  1992).  The  term  "case,"  however,  is  often  used  to  refer  to  everything  from  a  two- 
paragraph  scenario  at  the  end  of  a  textbook  chapter  to  an  extensive  history  with  detailed 
descriptions  of  events,  decisions,  and  people.  In  business  education,  cases  are  based  on 
public  financial  records,  management  interviews,  direct  observations,  and  corporate 
communications;  however,  the  wealth  of  data  from  all  these  sources  typically  is  filtered, 
distilled,  and  summarized  when  written  up  for  students.  What  results  is  typically  a  10  to 
30  page  narrative  document  that  may  also  include  organizational  charts,  tables  of  financial 
data,  resumes,  and/or  corporate  memos.  Thus,  typical  "Harvard-style"  cases  (Harvard  is  the 
largest  publisher  of  business  cases)  usually  lack  the  complexities  of  real  work  situations. 
Such  cases  may  not  allow  students  an  opportunity  to  develop  appropriate  problem-solving 
skills.  While  students  may  be  able  to  deal  with  neatly  packaged  problems  within  a  case, 
they  are  all  too  often  unable  to  deal  with  similar  problems  in  a  dynamic  workplace.  Another 
weakness  of  many  business  cases  is  that  they  are  written  to  illustrate  a  specific  concept, 
such  as  leadership,  marketing  strategies,  or  ethics.  In  limiting  their  scope,  these  cases  may 
not  adequately  help  students  to  see  the  interrelationships  of  these  concepts  in  the  real 
world.  In  analyzing  business  cases,  students  usually  approach  them  as  having  all  the 
information  needed  and  that  all  included  information  is  pertinent.  Cases  may  present 
puzzles,  but  students  assume  that  all  the  pieces  of  the  puzzles  are  there.  Often,  traditional 
cases  don't  require  students  to  go  beyond  the  information  given  or  to  integrate  other 
resources  into  their  case  analyses.  Another  problem  with  the  way  cases  are  typically  taught 
is  that  they  are  used  in  one  class,  discussed,  and  then  rarely  referred  to  again.  Few 
connections  are  made  across  cases. 

Based  on  our  study  and  practice,  we  recommend  the  creation  of  cases  that  offer  more 
experiential,  integrated,  and  authentic  opportunities  for  learning.  We've  called  such  cases 
"complex"  cases,  though  they  could  also  be  called  experiential  cases,  rich  cases,  enhanced 
cases,  or  case  experiences.  Complex  cases  are  cases  that  present  real  issues  and  problems 
for  analysis  via  real  materials.  The  materials  may  include  print,  video,  or  other  media. 
Complex  cases  use  media  to  enable  large  numbers  of  students  to  participate  in  many  of  the 
cognitive  activities  involved  in  actual  field  experiences.  In  a  complex  case,  critical 
information  is  embedded  in  an  authentic  context  of  documents,  reports,  memos, 
conversations,  and  other  organizational  artifacts.  Putting  information  into  neat  categories 
—  an  approach  that  overguides  students  to  a  "correct"  answer  —  is  avoided.  Thus,  complex 
cases  are  more  realistic  and  messy  than  traditional  cases.  While  complex  cases  present 
layers  of  detailed  'nformation,  they  avoid  intentionally  including  red-herrings,  statements 
that  lay  open  traps  for  particular  lines  of  faulty  thinking.  In  addition,  complex  cases  pose 
problems  that  have  many  possible  answers  and  that  can  be  looked  at  from  multiple 
viewpoints.    Complex  cases  necessitate  discussion  and  thus  should  be  group  activities. 
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Complex  cases,  in  essence,  compress  real  information  and  communication.  They  are 
most  like  field  research  in  that  to  be  successfully  completed  they  require  students  to  seek 
out  information,  determine  the  value  of  information  given  to  them,  and  analyze  and 
synthesize  their  knowledge  with  new  information.  In  a  typical  field  experience,  however, 
students  spend  months  gathering  the  types  of  company  documents,  memos,  and  interviews 
that  are  presented  in  a  complex  case.  While  presenting  these  within  a  complex  case  instead 
of  field  research  causes  students  to  lose  practice  in  consulting  with  companies,  many 
efficiencies  are  gained.  It  is  difficult  to  provide  field  experiences  for  students  on  a  regular 
basis  or  for  large  number  of  students.  In  addition,  many  students  do  not  have  adequate 
time  and  resources  for  a  semester-long  field  project  requiring  travel  and  other  expenses. 

The  goals  of  complex  cases,  much  like  the  goals  of  professional  education,  are 
ambitious.  The  key  cognitive  goals  of  using  complex  cases  are  to  provide  opportunities  for 
students  to  develop  flexible  structures  of  knowledge,  transition  from  novice  to  expert,  think 
more  like  professionals,  acquire  the  skills  and  language  of  their  field,  and  deal  with  ill- 
structured  information. 

As  business  students  build  the  advanced  knowledge  required  to  become  managers 
they  must  grapple  with  increasing  conceptual  complexity  and  ill-structuredness.  In  order  to 
grapple  with  this  successfully,  instructional  activities  should  foster  cognitive  flexibility  (Spiro, 
Feltovich,  Jacobson,  &  Coulson,  1991;  Spiro  &  Jehng,  1990;  Spiro,  Coulson,  Feltovich,  & 
Anderson,  1988).  Spiro  defines  cognitive  flexibility  as  the  ability  to  quickly  restructure  one's 
knowledge  in  response  to  radically  changing  situational  demands  (Spiro  &  Jehng,  1990). 
Cognitive  flexibility  is  achieved  through  avoiding  oversimplification,  striving  for 
interconnectedness  of  concepts,  promoting  multiple  representations,  and  active  learning. 

Spiro  et  al.  (1988,  1991)  claim  cognitive  flexibility  is  particularly  important  in  ill- 
structured  domains,  i.e.,  areas  of  knowledge  that  have  concept-  and  case-  complexity  as  well 
as  across-rase  irregularity.  They  note  that  the  more  ill -structured  the  domain,  the  poorer 
the  guidance  for  knowledge  application  that  "top-down"  structures  will  generally  provide. 
Such  structures  lose  their  usefulness  because  of  the  great  variability  from  case  to  case  and 
the  great  variability  in  which  the  same  concept  is  used  across  cases.  This  irregularity  works 
against  the  use  of  knowledge  structures  that  assume  "routinizability"  across  cases.  One  of 
Spiro's  major  arguments  is  that  one  has  to  'criss-cross'  complex  material  in  order  to 
understand  it  —  similar  to  the  way  one  has  to  criss-cross  a  landscape  in  order  to  get 
familiar  with  it.  By  criss-crossing  through  cases  and  concepts,  students  establish  many 
alternative  paths  to  get  from  one  part  of  their  knowledge  base  to  any  other  part.  Useful 
knowledge  must  be  learned,  presented,  and  tried  out  in  many  ways. 

Typical  professional  programs,  particularly  medical  programs,  have  been  criticized 
by  Spiro  et  al.  (Feltovich,  Spiro  &  Coulson,  1989)  for  oversimplifying  content,  being  too 
theoretical,  decontextualizing  concepts  and  principles,  and  heavily  compartmentalizing  the 
curriculum.  Such  tendencies  are  major  reasons  for  the  lack  of  transfer  of  knowledge  from 
university  settings  to  the  field.  That  is  not  to  say  that  all  learning  must  take  place  in  the 
field.  Collins,  Brown,  and  Newman  (1988)  introduced  the  notion  of  cognitive  apprenticeship 
which  states  that  the  learning  of  knowledge  and  skills  must  be  embedded  in  the  social  and 
functional  context  of  their  use.  There  is  an  emphasis  on  authentic  cognitive  processes  more 
than  authentic  physical  environments.  Complex  cases  attempt  to  use  these  understandings 
to  provide  students  with  rich,  authentic  tasks. 


Recent  research  on  expert-novice  distinctions  suggest  that  the  cognitive  schemata  of 
experts  are  more  elaborate,  more  complex,  more  interconnected,  and  more  easily  accessible 
than  those  of  novices  (Leinhardt  &  Greeno,  1986;  Chi,  M.,  Feltovich,  P.,  &  Glaser,  R.  1982). 
While  expertise  is  developed  through  continued  experience  with  a  wide  range  of  examples, 
case-based  instruction  can  simulate  some  of  this  experience  and  provide  problem-solving 
practice  for  students.  It  is  intended  that  through  complex  cases  students  not  only  will 
increase  their  experience  with  new  concepts  and  examples,  but  they  will  also  increase  their 
ability  to  see  the  interconnectedness  of  concepts  and  examples. 

Automotive  Armature:  An  Example  of  a  Complex  Case 

In  response  to  national  criticisms  of  accounting  education  as  well  as  research 
advocating  more  situated,  integrative,  and  active  learning  environments,  a  professor  at 
Indiana  University's  School  of  Business  undertook  the  challenge  of  creating  an  authentic, 
rich  case.  The  case  is  about  a  real  manufacturing  company,  Automotive  Armature  (AA),  that 
was  experiencing  a  variety  of  financial  difficulties  and  operational  problems.  The  AA  case 
was  originally  used  as  a  field  research  experience  for  a  class  of  25  undergraduate  students 
in  an  upper-level  accounting  class.  In  teams,  the  students  analyzed  AAs  accounting  system. 
They  found  the  experience  highly  motivating  and  that  working  with  an  actual  company 
made  accounting  concepts  and  business  issues  much  more  relevant  for  them.  While  on-site 
field  research  had  many  learning  benefits,  it  posed  logistical  problems  when  the  professor 
decided  to  integrate  AA  into  a  core  course  in  managerial  accounting  for  250  MBA  students. 
While  this  number  of  students  could  not  be  sent  to  the  company,  the  company  could,  at  least 
in  part,  be  brought  to  them. 

The  AA  case  was  designed  with  several  learning  and  technology  goals  in  mind. 
First,  the  case  was  intended  to  promote  students'  discovery  of  problems  in  a  complex, 
realistic  business  environment.  In  designing  the  case,  an  attempt  was  made  to  create  a 
learning  environment  that  contextualized  managerial  accounting  concepts  and  integrated 
traditionally  compartmentalized  business  curricula  (e.g.,  accounting,  marketing,  operations). 
Unlike  most  Harvard-style  cases,  the  AA  case  did  not  present  information  in  a  distilled, 
neatly  organized  manner.  It  was  essential  that  this  case  lead  students  to  experience  the 
process  of  identifying  important  information,  organizing  it,  and  analyzing  it,  rather  than 
having  the  case's  organization,  structure,  or  brevity  do  it  for  them.  The  students'  task  of 
determining  key  AA  problems  and  recommending  solutions  was  designed  to  be  highly 
authentic. 

On  the  technology  side,  the  AA  case  had  to  provide  textual,  numerical,  and  visual 
data  for  students  and  it  had  to  be  "implementable."  Forty  groups  would  be  working  with 
the  case,  so  the  technology  had  to  be  available,  accessible,  and  require  minimal  training. 
For  Spring  1993,  the  AA  case  resources  provided  to  the  MBA  students  consisted  of  the 
following: 

25-minute  overview  video  of  the  company 

Eight  10-  to  20-minute  video  interviews  with  the  Chief  Financial  Officer  (CFO) 

A  140-page  document  with  company  and  industry  data 

A  live,  one-hour  Q&A  session  with  the  CFO 

The  two  class  professors  who  served  as  case  mentors 

A  case  consultant  (an  accounting  MBA  student)  who  served  as  a  resource  person  for 
the  case 
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To  give  students  a  good  sense  of  the  company,  it  was  important  to  show  them  what  the 
company  looked  like,  how  its  products  were  produced,  what  the  working  conditions  were,  etc. 
An  overview  video  was  produced  to  do  this.  The  interview  videos  gave  students  more 
detailed  information  about  different  aspects  of  the  company  as  well  as  role-modeled  effective 
interviewing  techniques  for  them.  The  print  documents  provided  students  with  real 
company  information  via  spreadsheet  printouts,  plant  diagrams,  newspaper  articles, 
organizational  charts,  interoffice  memos,  reports,  and  other  artifacts. 

In  their  regular  classes  of  65  students,  the  MBAs  were  introduced  to  the  case  by  one 
of  the  professors.  It  was  stressed  to  them  that  this  case  was  intended  to  help  them  build 
their  critical  and  creative  thinking,  communication,  and  problem  solving  skills  as  well  as  to 
help  them  better  understand  the  role  of  managerial  accounting  in  conjunction  with  other 
company  functions.  Their  task,  as  explained  to  them  by  the  professor,  was  to  sort  through 
the  information  presented  in  the  case  resources,  identify  four  or  five  of  the  company's  key 
problems,  and  recommend  solutions  that  would  be  feasible  for  the  company  within  a  6-12 
month  time  frame.  Their  task  was  situated  in  the  context  of  a  consulting  engagement  in 
which  students  directed  their  recommendations  to  the  company's  Chief  Financial  Officer. 

This  was  a  team  project  in  which  students,  in  groups  of  five  or  six,  had  about  10 
days  to  go  through  the  case  materials,  determine  their  recommendations,  and  write  a  12- 
page  (plus  appendices)  group  paper.  While  the  members  of  each  team  shared  a  copy  of  the 
videotape,  everyone  purchased  their  own  copy  of  the  140-page  case  document.  Each  team 
began  the  case  by  watching  the  AA  videos  together,  usually  at  a  team  member's  home, 
discussing  the  videos,  mapping  out  an  initial  strategy  for  determining  recommendations, 
and  assigning  individual  responsibilities. 

Most  teams  chose  to  assign  tasks  on  the  basis  of  members'  expertise  (e.g.,  "the 
finance  guy"  was  assigned  to  focus  on  the  company's  financial  issues,  the  marketing  major 
on  marketing  issues,  etc.).  Throughout  the  next  stage  of  information  gathering  and 
organizing  the  students  worked  individually  and  in  groups,  regularly  putting  forth  ideas  and 
questions,  discussing  them,  and  refining  them.  The  final  stage  of  their  task  was  to  pull 
their  ideas  together  and  write  a  cohesive,  concise  paper.  In  their  group  papers,  students 
were  required  to  include  not  only  recommendations,  but  justifications  and  to  write  in  a 
business  format,  meaning  that  brevity,  clarity,  and  persuasiveness  were  key. 

Student  and  Professor  Reactions  to  the  AA  Case 

A  variety  of  data  has  been  collected  to  document  the  AA  case  activity  and  reactions 
to  it,  including  class  and  team  observations,  surveys,  and  interviews.  Overall,  both 
students  and  faculty  felt  that  the  AA  rase  provided  a  useful,  authentic  experience  that 
allowed  students  to  develop  their  analytical  skills. 

In  a  survey  after  the  AA  case  was  completed  (N=187),  most  students  indicated  that 
the  case  was  a  valuable  learning  experience  (75%)  and  that  it  would  be  worthwhile  to  do 
more  cases  like  it  (70°i>).  Kighty-five  per  cent  of  the  responding  students,  most  of  whom  had 
business  experience,  thought  that  the  AA  case  realistically  portrayed  a  business  situation 
and  that  this  case  was  more  interesting  than  the  Harvard-type  cases  they  had  done  (61%). 
They  also  thought  that  it  was  more  demanding  than  Harvard-type  cases  (82%).  While  41% 
of  the  students  indicated  that  they  were  overwhelmed  by  the  amount  of  information  in  the 
AA  case,  most  students  (72%)  said  they  were  able  to  make  sense  of  the  information.  The 
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largest  group  of  students  (41%)  reported  that  they  spent  21  to  30  hours  working  on  the  case, 
while  23%  reported  11-20  hours  and  17%  reported  31-40  hours. 

The  students  reacted  favorably  to  the  videos  and  appreciated  being  able  to  see  the 
company  and  its  facilities.  Many  students  thought  the  interviews  were  boring,  although 
they  acknowledged  that  they  contained  some  valuable  information.  Some  students  were 
frustrated  with  having  a  large  amount  of  seemingly  disorganized  data  dropped  in  their  laps. 
They  wanted  more  context  for  some  of  the  documents  in  the  written  materials,  e.g.,  where 
they  came  from,  why  a  consultant  might  request  such  a  document,  etc.  Given  the  time 
pressure  of  the  case,  most  teams  worked  in  "survival  mode."  They  divvied  up  tasks  and 
focused  on  getting  the  required  paper  completed.  There  was  not  a  great  deal  of  time  for 
reflecting  on  different  perspectives,  revisiting  concepts,  or  revising  their  papers.  From  our 
group  observations  it  was  evident  that  a  variety  of  problem-solving  strategies  were  used. 
Students  worked  in  their  teams  to  figure  out  what  information  was  most  relevant,  what 
information  was  missing,  how  they  might  get  access  to  missing  information,  and  how  they 
could  cross-validate  different  sources  of  information.  Most  teams  ran  additional  calculations 
of  financial  data  to  project  the  impact  of  their  recommendations.  Some  groups  even 
consulted  with  sources  such  as  a  city  chamber  of  commerce  and  an  accounting  firm  to  gather 
additional  information. 

The  professors  were  generally  pleased  with  the  results  of  the  AA  case,  though  they 
had  a  variety  of  suggestions  about  how  to  do  things  differently  the  next  time.  Two  primary 
suggestions  were  to  allow  more  time  for  the  case  and  to  put  it  later  in  the  semester  after  the 
concept  of  activity-based  costing  had  been  discussed.  They  felt  that  the  students'  papers 
were  good,  especially  given  the  time  constraints,  though  only  a  few  were  truly  innovative. 
They  noted  that  there  wasn't  a  wide  range  in  the  teams'  recommendations,  with  the  key  AA 
problems  discussed  in  most  papers  being  product  proliferation,  overly  liberal  core  and 
warranty  policies,  poor  inventory  tracking,  lack  of  product  costing,  and  excessive  inventory. 
While  some  of  the  recommendations  were  interrelated,  most  papers  broke  down 
recommendations  into  production,  marketing,  accounting,  systems,  or  transportation  issues. 
One  professor  commented  that  several  papers  failed  to  cover  basic  issues,  such  as  cash  flow. 
While  these  papers  may  have  discussed  related  issues,  they  failed  to  adequately  tie  the 
issues  together, 

Designlssues  concerning  Complex  Cases 

Cases,  in  general,  are  considered  useful  instructional  activities  for  the  development 
of  higher  level  skills.   Complex  cases  —  case  experiences  as  opposed  to  a  case  studies  — 
may  be  most  appropriate  when  advanced  problem-solving  skills,  expert  level  of 
performance,  and  integration  of  concepts  are  desired.  But  how  does  one  design  a  complex 
case?  When  creating  rich  learning  environments  for  students,  case  developers  (in  our  project, 
a  team  of  instructional  designers  and  subject  matter  experts)  are  confronted  with  several 
challenges.  What  should  be  included  in  the  case?  What  should  be  excluded?  How  should  it 
be  presented?  What  should  students  be  asked  to  do  with  the  case?  How  much  guidance 
should  they  be  given?  How  will  students'  learning  be  evaluated?  Three  major  dynamics 
should  be  considered  when  designing  a  complex  case:  authenticity,  organization  and 
duration  (see  Figure  1). 
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Efgure  1 

Dynamics  to  Consider  when  Designing 
Complex  Cases 


Authenticity 

A  high  degree  of  authenticity  is  essential  in  a  complex  case,  both  in  the  case 
materials  and  in  the  case  task.  This  can  be  achieved  by  creating  tasks  for  students  that  are 
as  close  to  field  experiences  as  possible  rather  than  fabricated  experiences.  Developers 
should  use  real  situations  for  their  cases  not  fictitious  ones,  raw  data  as  opposed  to 
narratives,  and  actual  documents  more  than  ummaries.  In  order  to  create  a  case  that  is 
demanding  and  engrossing,  case  developers  must  put  students  in  the  middle  of  dynamic, 
multifaceted  scenarios  that  reflect  the  natural  complexities  of  the  real  world. 

Multiple  sources  of  information  are  required  to  develop  a  complex  case.  To  get  the 
authenticity  and  richness  desired  in  a  complex  case,  case  developers  need  to  work  directly 
with  the  entity  that  is  the  subject  of  the  case,  e.g.,  a  company,  agency,  school,  hospital, 
group,  etc.  That  organization  must  provide  a  good  portion  of  the  case  information  and  be 
willing  to  allow  developers  to  sift  through  organizational  documents  and  communications. 
The  amount  and  quality  of  the  data  gathered  is  greatly  dependent  on  who  the  key  contact  is 
with  the  organization.  Additional  information  for  complex  cases  may  come  from  professional 
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reports,  newspaper  or  magazine  articles,  research  databases,  or  interviews  with  company 
clients  or  competitors.  In  the  AA  case,  access  to  the  company  grew       of  a  consulting 
engagement  that  a  professor  had  with  the  company.  Since  the  company  was  only  45 
minutes  away  from  the  university,  it  was  feasible  to  make  on-site  visits  to  meet  with 
company  personnel,  shoot  video  of  the  plant,  and  attend  company  meetings.  Having  close 
proximity  to  the  case  source  is  not  only  beneficial  for  these  reasons,  but  for  motivating  and 
involving  students  as  well.  In  the  AA  case,  knowing  that  the  company  was  nearby  helped 
increase  the  realism  and  relevance  of  the  case. 

In  designing  a  complex  case,  students  must  be  positioned  as  active  participants  in 
the  case,  not  passive  readers.  Accordingly,  the  usual  narrative  style  of  most  cases  that 
creates  a  linear  story  must  be  substantially  replaced  by  authentic  documents.  *\s  much  as 
possible,  the  language  of  the  field  should  be  used.  Thus,  spreadsheets  and  financial  reports 
were  used  in  the  AA  case.  Children's  drawings  or  test  norms  might  be  used  in  a 
psychodiagnostic  case  or  blue  prints  and  building  models  in  an  architecture  case. 

Case  developers  must  also  determine  how  the  case  will  be  presented  to  students. 
Typically,  cases  are  paper-based.  This  is  economical  and  easily  distributed.  In  a  complex 
case  there  is  an  effort  to  create  a  rich  environment  that  has  as  high  a  fidelity  with  reality  as 
possible.  Thus,  the  use  of  video  is  recommended  to  bring  the  field  environment  into  the 
classroom  environment.  Video  adds  another  level  of  richness  to  the  data  students  must 
work  with  in  analyzing  a  case.  Via  video  students  can  see  a  company's  facilities,  personnel, 
working  conditions,  and  products.  The  use  of  video  in  the  AA  case  was  seen  as  important 
because  it  would  allow  students,  like  consultants  in  the  field,  to  learn  about  the  company 
through  casual  visual  information,  e.g.,  dirtiness  of  used  parts,  disorganization  of  barrels, 
mannerisms  of  management,  etc. 

Organization 

Even  a  rich,  complex  case  must  have  some  sort  of  organization.  The  act  of  capturing 
a  piece  of  reality  and  presenting  it  to  students  unavoidably  involves  structuring  that  reality. 
Case  developers  must  determine  the  kind  and  amount  of  structure  to  give  the  case,  yet  at 
the  same  time  avoid  imposing  undue  analysis  or  interpretation  to  the  materials.  A  flexible 
organization  maximizes  the  students'  challenge  to  determine  their  own  organization  and 
priorities  for  the  case.  An  organization  that  is  too  structured  tends  to  turn  information  into 
answers  and,  therefore,  is  less  challenging  and  less  authentic.  Many  organizing  schemes  are 
possible:  chronological,  alphabetical,  hierarchical,  functional,  or  topical.  Usually  it  is  best  to 
choose  a  scheme  that  is  authentic  to  the  case  environment.  For  example,  in  the  AA  case  we 
organized  the  video  interviews  and  print  materials  around  the  typical  functions  of  a 
business;  accounting/finance,  management,  production,  marketing,  and  human  resources. 

The  next  organizational  challenge  is  to  determine  the  boundaries  of  the  case;  what 
to  include,  exclude,  summarize,  etc.  If  one  decided  to  "include  everything"  a  complex  case 
would  be  huge  to  the  point  of  being  unmanageable.  While  a  complex  case  should  have  broad 
rather  than  narrow  boundaries,  case  developers  have  to  make  some  decisions  about  what  to 
leave  out.  Several  factors  guide  such  decisions.  First  of  all,  what  is  the  level  of  the  learners? 
Freshmen  may  be  less  able  to  deal  with  masses  of  unstructured,  detailed  information  than 
graduate  students.  The  less  advanced  learners  are,  the  more  developers  should  consider 
summarizing  lengthy  documents,  omitting  tangential  materials,  limiting  the  number  of 
issues  in  a  case  and/or  allowing  more  time  for  students  to  work  with  the  case.  More 
important  than  the  case  materials  themselves,  however,  is  the  level  of  challenge  in  the 
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students'  task  for  the  case  activity.  A  complex  case,  including  the  data  net  from  which  it  is 
derived,  may  be  usable  across  a  variety  of  student  levels  provided  the  learning  goals  arc*  put 
at  an  appropriate  level.  When  determining  the  organization  of  a  complex  cc  a  the 
boundaries  shouldn't  be  limited  to  a  particular  area,  for  example  in  the  business  field  to  j.ist 
management,  operations,  or  marketing.  While  a  complex  case  might  be  used  in  one 
particular  class,  it  should  include  perspectives,  issues  and  data  from  other  "classes,"  i.e., 
areas  or  related  perspectives,  to  give  it  realistic  richness. 

Duration 

A  third  dynamic  to  consider  when  designing  complex  cases  is  duration,  both  in  terms 
of  the  amount  of  time  covered  about  the  organization  described  in  the  case  and  the  amount 
of  time  that  students  are  intended  to  work  on  the  case  project.  The  time  period  described  in 
a  case  could  be  a  week  or  a  decade  as  long  as  that  time  period  contains  enough  rich 
information  to  engage  students  in  a  "ariety  of  issues.  In  the  AA  case  we  provided  some 
general  historical  information  about  AA,  but  primarily  concentrated  on  the  company's  most 
recent  fiscal  year  (a  typical  framework  for  a  business  analysis).  Originally,  we'd  intended 
that  the  AA  case  be  used  throughout  the  semester,  but  a  variety  of  curriculum  logistics 
prevented  that.  The  students  ended  up  spending  approximately  10  days  on  the  AA  case 
which  was  later  considered  too  short  a  period  of  time  to  achieve  all  the  goals  of  the  case 
activity.  Complex  cases  are  time  consuming.  Time  must  be  allowed  for  siudents  to  visit 
and  revisit  the  many  different  case  resources,  to  "criss-cross  the  landscape." 

Learning  Issues  concerning  Complex  Cases 

Complex  cases  place  a  high  learning  demand  on  students,  more  so  than  with 
traditional  cases.  Three  of  the  key  challenges  that  students  face  are  dealing  with 
information  overload,  coping  with  a  high  level  of  ambiguity,  and  integrating  information  from 
multiple  perspectives. 

Dealing  with  Information  Overload 

When  presented  with  lots  of  "raw  data,"  most  students  are  initially  overwhelmed. 
"Where  do  I  start?"  and  "How  will  I  make  sense  of  all  this?"  are  typical  questions  running 
through  their  heads.  From  a  learning  perspective,  students  must  find  ways  to  organize  the 
data  in  meaningful  ways.  We  recommend  that  students  start  in  groups,  scanning  the  case 
documents  and  other  materials  and  making  some  initial  hypotheses  about  important  issues 
or  problems.  What  was  once  a  mass  of  data  now  becomes  data  that  is  focused  by  specific 
problems.  With  a  problem-based  focus,  students  can  more  effectively  sort  useful  from  the 
less-useful  information  that  is  presented  in  a  complex  case.  Depending  on  their  problem 
focus,  one  student's  information  may  be  another  student's  non-information.  As  students 
read  or  view  case  materials,  they  examine  the  materials  in  light  of  problems  and  hypotheses 
about  their  solutions.  It  is  essential  that  students  discuss  their  hypotheses  with  others  as 
they  delve  further  into  the  case  to  both  deal  with  information  overload  and  to  check  their 
hypotheses.  Through  this  discussion  new  hypotheses  may  emerge.  The  abundance  of 
information  in  a  complex  case  can  also  be  managed  by  asking  questions.  The  professor  or 
ense  consultant  can  be  useful  resources  in  helping  students  pose  and  focus  effective 
questions. 
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Coping  with  a  High  Level  of  Ambiguity 


A  complex  case  presents  a  scenario  (or  multiple  scenarios)  that  does  not  lead 
students  to  a  predetermined  end.  Students  can  take  a  complex  case  in  many  directions. 
Their  task  is  somewhat  ambiguous  and  lacks  a  precise  algorithm  to  follow.  When  students 
are  asked  to  solve  problems  in  more  traditional  cases,  they  know  the  problems  are  within 
the  boundaries  of  the  case  and  more  often  than  not  focused  on  that  week's  topic  in  class. 
The  danger  in  this  is  that  the  case  may  be  no  more  than  an  illustration  of  a  problem  rather 
than  a  challenging  intellectual  activity.  In  a  complex  case  the  students'  task  is  one  of 
problem  finding  as  well  as  problem  solving.  Ambiguity  is  a  necessary  component  of  a 
complex  case  because  it  is  frequently  a  difficult  aspect  of  professional  work  as  well. 
Students  have  to  struggle  with  the  ambiguity  in  order  to  reach  the  higher  learning  goals. 
While  this  struggle  is  necessary,  it  may  be  appropriate  to  help  students  anticipate  this 
struggle  and  acknowledge  the  difficulties  of  dealing  with  ambiguity.  The  tasks  involved  in  a 
complex  case  activity  may  be  very  different  from  the  activities  in  students'  other  classes  so 
coaching  students  in  new  ways  of  working  may  be  necessary. 

Integrating  Information  from  Multiple  Perspectives 

As  students  read  a  complex  case,  they  gather  and  organize  information  from  a 
variety  of  sources.  As  active  participants  in  the  case,  students  choose  to  go  down  many 
different  paths  that  they  may  also  choose  to  cross  and  re-cross.  The  students'  challenges  are 
to  identify  and  assess  different  perspectives  and  to  integrate  them  in  order  to  make  effective 
recommendations.  The  materials  in  a  complex  case  should  give  students  many 
opportunities  to  examine  the  same  information  repeatedly  from  different  perspectives.  To 
help  students  integrate  information  from  multiple  perspectives,  they  must  be  given 
opportunities  to  discuss  these  perspectives.  Such  discussion  should  include  "what  if  or 
devil's-advocate  style  questions  to  deepen  students'  understanding  of  different  perspectives. 
In  addition,  it  is  helpful  to  integrate  concepts  from  a  complex  case  into  other  discussions.  In 
the  AA  case  this  occurred  both  naturally  and  intentionally.  The  MBA  students  often  brought 
up  Automotive  Armature  as  an  example  in  class  discussions  after  the  case  activity.  The 
class  professors  also  used  the  AA  case  again  at  the  end  of  the  semester  for  an  activity-based 
costing  assignment. 

Implementation  Issues  concerning  Complex  Cases 

Many  issues  must  be  considered  and  addressed  when  implementing  complex  cases. 
Three  of  the  most  important  will  be  discussed  here.  The  first  revolves  around  the  concept  of 
appropriation,  i.e.,  the  process  by  which  the  design  and  the  product  are  actually  used  by  the 
professor  and  the  students.  The  second  implementation  issue  is  how  the  complex  case  fits 
within  the  larger  educational  environment.  A  third  major  issue  concerns  the  instructor's 
perspective  on  teaching. 

Appropriation 

Complex  cases  are  intentionally  designed  to  have  a  great  deal  of  flexibility.  This 
flexibility  promotes  appropriation.  Case  developers  should  be  willing  to  allow  users  to 
assimilate  the  case  materialR  and  activities  into  their  current  ways  of  doing  things,  even  if 
this  means  changing  them  from  their  original  intent.  All  instructional  products  are 
appropriated  in  some  way  by  the  end  umrn.  For  example,  group  work  may  actually  be  done 


by  individuals  outside  of  class;  a  case  designed  for  a  month  may  last  a  week;  information 
designed  for  random  computer  access  may  be  printed  out  and  read.  This  actual  use  is 
fundamentally  outside  of  the  control  of  the  designers.  One  of  the  traditional  complaints 
within  instructional  design  is  that  the  designs  are  excellent  but  they  are  never  "correctly" 
implemented.  This  frustration  has  led  to  the  notions  of  getting  rid  of  the  teacher  and  that 
delivery  through  technology  alone  is  preferable.  But  appropriation  is  a  natural,  and  not 
necessarily  negative,  process.  By  observing  how  professors  and  students  appropriate 
complex  cases,  developers  can  gain  insight  about  how  to  improve  the  case  materials  and 
activities  and  how  to  build  tools  that  capitalize  on  their  natural  use. 


Fit 

The  second  major  issu*)  in  implementing  complex  cases  involves  how  the  case  fits 
within  the  overall  educational  environment.  How  a  case  fits  in  with  the  rest  of  students' 
curriculum  and  educational  demands  affects  the  implementation.  Complex  cases  should  be 
related  to  at  least  other  topics  within  the  course  if  not  multiple  courses  in  a  curriculum. 
They  should  not  be  seen  as  disconnected  activities  that  have  nothing  to  do  with  other  things 
students  are  learning.  Complex  cases  require  group  work.  Teams  may  or  may  not  be  a  part 
of  the  educational  culture.  If  not,  steps  should  be  taken  to  introduce  students  to  this  way  of 
working  and  to  provide  extra  support  for  this  during  tlie  case  activity.  Complex  cases  are 
very  time  demanding,  so  they  should  be  scheduled  at  *  time  when  students  have  fewer 
demands  in  their  other  classes  or  they  should  be  scheduled  for  incremental  use  over  a  longer 
period  of  time.  In  the  AA  case  activity,  it  turned  out  that  some  student  teams  had  major 
presentations  in  another  course  and  many  students  were  worried  about  their  upcoming  mid- 
terms. Thus,  some  students  did  not  devote  sufficient  time  to  the  AA  case  activity  and 
allowed  their  fellow  team  members  to  pick  up  the  slack. 

Instructor's  Perspectives 

A  third  major  issue  in  implementing  complex  cases  is  the  instructor's  perspective  on 
teaching  in  terms  of  having  any  one  "right"  answer  to  the  case.  Complex  cases  are 
considered  to  have  a  wide  array  of  possible  right  answers.  The  amount  of  information  and 
context  is  broad  enough  to  allow  for  a  large  number  of  perspectives  to  be  brought  out  as  key 
concepts.  This  should  allow  for  the  possibility  of  the  case  being  reused  frequently  with  a 
change  of  student  goal.  If,  however,  the  general  attitude  of  students,  professors,  or  the 
program  in  general  is  that  cases  have  singular  correct  answers  or  are  about  single  issues, 
the  complex  case  may  not  be  used  frequently  enough  to  be  cost-effective.  It's  important  that 
the  case  be  designed  flexibly  enough  that  it  not  focus  on  a  single  issue.  This  way,  the  case 
provides  sufficient  information  and  context  so  that  the  activity  can  be  substantively  different 
each  semester  by  changing  the  basic  question.  In  this  project,  the  same  case  materials  were 
utilized  twice  in  the  semester  by  changing  the  focus  of  the  problem  and  adding  a  limited 
amount  of  additional  case  materials. 

A  related  issue  with  complex  cases  is  the  instructor's  perspective  on  teaching  in 
terms  of  being  an  nuthority.  He  or  she  must  be  willing  to  accept  a  teaching  role  that  is  one 
of  a  guide  rather  than  a  director.  With  complex  cases,  the  instructor  must  hold  back  and 
allow  students  to  find  their  own  answers  rather  than  telling  them  possible  answers.  It  is 
important  for  the  exper.  —  whether  that's  the  professor,  the  company  visitor  or  the 
advanced  student  —  to  be  careful  of  revealing  too  much  to  the  students  and  to  provide  a 
framework  that  stimulates  students  to  find  things  out  for  themselves. 
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Some  Caveats  //or  creating  Electronic  or  Multimedia  Complex  Cases 

A  major  point  for  implementing  complex  cases  is  to  realize  that  they  are  not  just  for 
individuals.  A  common  problem  of  utilizing  an  electronic  or  multimedia  format  requiring 
computers  is  that  many  people  still  see  computers  primarily  for  individual  activity. 
Computer  laboratories  are  set  up  with  individual  workstations.  People  have  individual  and 
personalized  laptop  or  desktop  computers.  More  often  than  not,  converting  learning 
materials  into  a  computer  format  may  itself  contribute  to  a  breakdown  of  team  cooperation 
in  the  analysis  and  problem  solving  of  rich  cases. 

Our  recommendation  is  for  a  specific  component  designed  to  start  the  team 
discussion.  In  the  AA  case,  the  video  —  which  early  in  the  design  phase  was  seen  as 
reference  material  —  ended  up  as  a  vehicle  for  interaction  of  the  student  team  members. 
Each  team  received  a  videotape  and  they  were  encouraged  to  watch  it  the  first  time  together. 
Certainly,  a  multimedia  version  of  the  case  with  random  access  to  the  video  at  a  later  time 
could  be  useful  to  students.  However,  the  video  as  a  social  vehicle  for  examining  the  case 
and  discussion  case  issues  was  probably  more  valuable. 

Another  major  point  of  design  in  a  complex  case  is  to  make  sure  that  the  materials 
are  readily  accessible.  Here  again,  we  confront  the  dilemma  that  turning  a  case  into  an 
electronic  format  may  make  it  less  accessible,  not  more  so.  In  the  AA  case,  a  thick  stack  of 
3-hole  punched  papers  were  the  print  materials  provided  to  all  students.  Students  put 
these  papers  in  binders  where  they  could  write  on  the  documents,  rearrange  them,  highlight 
them  or  index  them.  If  we  had  put  the  AA  case  into  a  computer  format  it  would  have 
required  a  high-end  desktop  computer  which  would  nbt  have  been  broadly  available  either 
at  the  university  or  by  the  students  at  home.  Several  stations  in  the  MBA  computer  lab 
could  have  had  the  AA  case  on  them,  but  this  would  not  have  been  enough  access  for  250 
students.  Without  appropriate  accessibility,  an  electronic  format  may  discourage  people 
from  the  type  of  reading,  review  and  reflection  of  the  materials  necessary  for  the  problem- 
solving  nature  of  the  activity. 

Conclusions 

There  are  few  examples  of  what  we  have  been  calling  complex  cases.  More 
instructional  activities  based  on  complex  cases  need  to  be  developed  and  researched  to  gain 
a  clearer  understanding  of  of  their  potential  and  optimal  uses.  As  an  example  of  a  complex 
case,  the  Automotive  Armature  case  had  mixed  success.  While  the  students  and  professors 
found  it  more  valuable  than  traditional  cases,  there  were  still  problems  with  students 
compartmentalizing  concepts  and  not  integrating  their  recommendations.  However,  we  still 
feel  that  complex  cases  are  a  step  in  the  right  direction  in  getting  students  to  develop  flexible 
structures  of  knowledge.  C    complex  case  alone  cannot  be  expected  to  create  an  expert.  It 
is  a  slow  and  difficult  process  to  develop  knowledge  and  skills  in  ill-structured  domains 
(Spiro  &  Jehng,  1990).  Students  may  well  benefit  from  a  series  of  experiences  —  in  the  field 
when  possible  and  through  complex  cases  when  not  —  that  allow  for  exploration  and 
learning  without  compromising  the  fundamental  complexity  of  ill-structured  domains  within 
profeHsional  disciplines. 
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Many  writers  suggest  that  one  should  employ  metaphors  in  creating  hypermedia/ 
multimedia  products.  For  example,  Erickson  (1990)  suggested  that  "metaphors  function  as 
natural  models,  allowing  us  to  take  our  knowledge  of  familiar,  concrete  objects  and 
experiences  and  use  it  to  give  structure  to  more  abstract  concepts"  (p.  66). 
Rosendahl-Kreitman  (1990)  suggested  that  "metaphors,  if  they  are  appropriate,  are  quickly 
understood"  and  thus  help  to  reduce  the  extent  to  which  users  need  to  learn  to  use  the 
system  completely  from  scratch  (p,  1.7).  Erickson  (1990,  p.  72)  accounted  for  this  "reduced 
learning  curve"  by  suggesting  that  a  metaphor,  once  adopted,  brings  with  it  "additional  bits 
of  structure  that  may  be  useful  later  on,"  a  suggestion  that  finds  support  in  the  work  of 
Allinson  and  Hammond  (1989),  Mifcxh  and  Dubberly  (1990)  and  Thimbleby  (1984).  Despite 
these  proposed  benefits,  Semper  (1990),  Rosendahl-Kreitman  (1990),  and  Vertelney,  Arent, 
and  Lieberman  (1990),  among  others,  have  warned  that  inappropriate  metaphors  can  cause 
confusion,  misunderstanding,  and  both  navigational  and  conceptual  difficulties  for  users. 

What  is  unclear,  however,  is  how  one  can  identify  an  "appropriate"  metaphor.  What  are  the 
key  properties  of  a  metaphor  and  how  exactly  does  one  operate?  What  makes  one 
metaphor  more  appropriate  than  another?  Is  more  than  one  metaphor  appropriate  for  a 
given  product?  Can  we  use  multiple  metaphors  in  a  product?  If  so,  how  do  we  determine  if 
they  interact  in  ways  that  contribute  to  the  effectiveness  of  the  product?  This  paper  seeks  to 
address  these  questions.  It  begins  by  examining  what  a  metaphor  is  and  how  it 
operates.  Next,  it  divides  metaphors  into  two  classifications  and  offers  an  approach  for 
analyzing  and  selecting  metaphors.  Several  metaphors  are  then  analyzed  and  methods  for 
evaluating  the  effectiveness  of  the  selected  metaphors  proposed.  The  pi^er  closes  by 
considering  whether  employing  metaphors  is  worth  the  effort. 

WHAT  A  METAPHOR  IS  AND  HOW  IT  OPERATES 

Lakoff  and  Johnson  (1980)  defined  the  essence  of  metaphor  as  "understanding  and 
experiencing  one  kind  of  thing  in  terms  of  another"  (p.  5),  A  metaphor  compares  one  thing  to 
another  without  directly  stating  the  comparison  and  "the  likeness  is  suggested  by  terms  not 
literally  applicable  to  each  other"  (Walsh  &  Walsh,  1972,  p.  165).  Mac  Cormac  (1985) 
described  metaphor  in  terms  of  mental  processes.  He  suggested  that  metaphor  is  "the 
creative  cognitive  process  of  .ctivating  widely  separated  areas  of  long-term  memory  and  of 
combining  normally  unassociated  concepts'*  (p.147). 

I  low  exactly  does  the  process  of  comparison  occur?  What  brings  together  Mac  Cormac 's 
"widely  separated  areas"  and  "normally  unassociated  concepts"?  Mac  Cormac  argued  that 
"those  concepts  are  juxtaposed  in  a  meaningful  manner  because  at  least  some  of  the 
semantic  features  of  each  concept  are  similar  to  one  another"  (p.  147).  Mountford  (1990) 
agreed,  suggesting  that  metaphors,  through  comparison,  convey  "both  superficial  and  deep 
siur'/tritirs  between  familiar  and  novel  situations"  (p.  25).  The  interaction  of  similarities 
mid  differences,  therefore,  is  the  key  to  the  effectiveness  of  a  metaphor. 

This  seems  closely  allied  with  Black's  (1979)  division  of  the  metaphor  into  two  components, 
the  primary  subject  and  the  secondary  subject.  Black  argued  that  metaphors  work  by 
projecting  upon  the  primary  subject  'associated  implications'  drawn  from  the 
characteristics  and  uses  of  the  secondary  subject.  According  to  Searle  (1979),  unco  we 
identify  something  as  a  metaphor,  we  examine  the  secondary  subject  to  identify  its 
properties  or  values  and  then  look  for  applicability  to  the  primary  subject.  The  impact  of  a 
metaphor  arises  from  the  interplay  of  these  components  (Kittay,  1987).  Kittay  suggested 


further  that  there  is  tension  between  the  two  components  of  a  metaphor  because  Black's 
*  associated  implications'  may  or  may  not  seem  appropriate  to  the  primary  subject. 


This  tension  seems  the  foundation  of  Miller's  (1979)  three-step  process  in  understanding 
metaphor:  recognition,  reconstruction,  and  interpretation.  In  the  recognition  step,  we 
perceive  the  metaphor  to  be  a  statement  of  relation  or  comparison  and  not  a  statement  of 
fact.  In  reconstruction,  we  start  to  rebuild  the  primary  subject  in  terms  of  the  properties  of 
the  secondary  subject  and  to  interpret  how  the  reconstructed  primary  has  changed  in  terms  of 
properties  and  operations.  According  to  Miller,  we  cannot  separate  reconstruction  and 
interpretation  because  many  possible  reconstructions  exist,  some  of  which  fit  the  context  and 
the  user's  intent  and  some  do  not.  Interpretation  and  reconstruction  interact  to  help  us 
identify  and  discard  inappropriate  reconstructions  and  retain  and  refine  reconstructions  that 
seem  suited  to  the  context  but  still  have  non-applicable  properties.  Thus,  as  Searle  (1979) 
argued,  we  limit  the  number  of  applicable  properties  to  those  which  we  judge  most 
appropriate.  Lakoff  and  Johnson  (1980)  and  Mountford  (1990),  among  others,  stressed  the 
importance  of  this  recursive  refinement  process  and  suggested  that  this  interactive  process 
plays  a  major  role  in  reshaping  an  individual's  understanding  of  the  metaphor. 

Kittay  (1987)  categorized  properties  of  the  secondary  subject  as  having  either  affinity  or 
contrast.  Properties  of  the  secondary  subject  that  are  appropriate  to  intent  and  context 
display  affinity,  while  properties  that  do  not  appear  to  match  intent  and  context  display 
contrast.  A  property  of  the  secondary  subject  that  exhibits  contrast  is  not  automatically 
excluded,  however.  Instead,  such  contrast  leads  the  individual  to  char  ge  his  or  her  "frame  of 
mind*';  that  is,  to  modify  expectations  and  interpretations  as  part  of  the  recursive  refinement 
process.  According  to  Kittay,  multiple  contrast  sets  exist  and  each  provides  a  set  of 
contrasting  properties  to  consider.  One  might  view  these  contrast  sets  as  a  semantic  field  of 
interconnected  and  associated  words  and  concepts.  This  description  of  how  a  metaphor 
operates  seems  evocative  of  Erickson's  (1990)  definition  of  a  metaphor  as  "an  invisible  web  of 
terms  and  associations  that  underlies  the  way  we  speak  and  think  about  a  concept''  (p.  66). 

Paivio  (1971)  considered  metaphor  in  light  of  what  he  termed  referential  meaning  and 
associative  meaning.  Referential  meaning  describes  a  relation  between  an  image  and  a 
verbal  label.  In  this  relation,  the  verbal  label  and  the  image  evoke  one  another  clearly. 
They  are  matched  exactly,  one  to  the  other.  Associative  meaning  alludes  to  groupings  of 
associations  involving  words,  images,  or  both  (pp.  84-85).  According  to  Paivio,  the  semantic 
system  and  the  imaginal  system  operate  both  separately  and  cooperatively.  As  the 
properties  of  primary  and  secondary  subjects  are  considered  in  the  recur?:  -  2  refinement 
process,  spread  of  activation  occurs  in  both  systems.  Both  he  and  Kosslyn  (1983)  argued 
that  much  of  our  cognitive  "knowledge"  is  actually  dual  coded.  It  exists  in  related  forms  in 
both  the  semantic  and  imaginal  systems.  Metaphor,  according  to  Paivio  (1979),  may  be  a 
particularly  effective  way  of  stimulating  interaction  between  the  semantic  and  imaginal 
systems.  Paivio  (1986)  used  the  term  referential  processing  to  refer  to  the  stimulation  of  one 
system  by  a  stimulus  in  the  other  system.  Associative  processing,  on  the  other  hand,  is  the 
process  by  which  a  stimulus  subsequently  stimulates  other  items  within  the  same  system. 
The  two  processes  constitute  the  spread  of  activation  within  and  across  the  two  systems. 
These  interconnections  between  systems  "are  not  assumed  to  be  one-to-one  but,  rather  one* 
to-many,  in  both  directions"  (Paivio,  1979,  p.  63). 

Images  may  be  more  easily  modified  in  the  refinement  process  than  are  semantic  items 
(Ambron,  1990;  Bugelski,  1971).  That  is,  the  spread  of  activation  within  and  across 
systems  may  enable  one  image  to  evoke  another  in  a  constantly  changing  pattern  of 
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relations  and  comparisons,  de  Bono  (1967)  labeled  this  property  of  imaginal  modification 
"fluidity  and  plasticity"  (p.  83)  and  suggested  that  one  could  modify  images  and  their  related 
properties  more  quickly  than  one  could  make  similar  modifications  in  verbal  labels  or 
expressions.  According  to  Gibbs  (1992),  our  mental  schemata  reflect  metaphorical  structures 
based  on  common  cultural  and  linguistic  experience*.  We  recognize,  without  formal  thought, 
that  a  particular  ir^taphor  reflects  these  metaphorical  structures,  and  our  unconscious 
recognition  stimulates  us  to  focus  on  related  associations  within  those 
structures. 

THE  RELATIONSHIP  BETWEEN  METAPHOR  AND  GRAPHICAL  USER 
INTERFACE 

If,  as  authors  cited  above  have  contended,  a  metaphor  helps  users  -  particularly  novice 
users  -  understand  how  a  hypermedia/multimedia  product  works,  then  how  is  the 
metaphor  implemented?  It  is  clear  how  writers  of  hooks  and  ?  m  makers  employ  metaphors. 
They  can  write,  "He  was  a  bear  of  a  man,'  or  state  that  "politicians  are  vultures  feeding  on 
lost  herds  of  inattentive  voters."  Although  computer  programs  are  not  necessarily  written  or 
spoken  expressions,  they  are  capable  of  using  metaphors  in  the  content  they  deliver. 
Hypermedia/multimedia's  more  likely  use  of  metaphor,  however,  is  as  visual  metaphor.  The 
vehicle  by  which  they  convey  visual  metaphors  is  most  frequently  the  interface.  Today's 
interfaces  are  often  graphical.  That  is,  they  employ  pictures  and  illustrations  that  make  the 
screen  and  interface  appear  to  be  something  other  than  a  blank  screen  on  which  text  or 
illustr  ations  appear.  Graphical  user  interfaces  (GUIs)  often  employ  icons  (little  graphical 
images  or  pictures)  to  represents  options  and  actions.  They  also  often  employ  a  general 
screen  appearance  that  may  resemble  something  other  than  a  computer  screen.  These  visual 
metaphors,  then,  represent  the  use  of  metaphor  in  hypermedia/multimedia  products. 

TWO  CLASSES  OF  METAPHORS 

For  purposes  of  discussion,  let  us  divide  metaphors  used  in  hypermedia/multimedia 
products  into  two  classes:  underlying  (or  primary)  metaphors  and  auxiliary  (or  secondary) 
metaphors.  An  underlying  metaphor  is  the  principal  or  first  metaphor  employed,  while  an 
auxiliary  metaphor  is  a  subsequent  metaphor  employed  by  the  product.  Some 
hypermedia/multimedia  products  may  employ  only  underlying  metaphors,  while  other  may 
employ  one  or  more  auxiliary  metaphors  in  addition  to  an  underlying  metaphor.  The  trick,  it 
would  seem,  is  to  select  auxiliary  metaphors  that  are  consonant  with  the  underlying 
metaphor.  In  the  words  of  Kittay  (1987),  we  need  to  look  for  auxiliary  metaphors  that 
exhibit  a  fairly  high  measure  of  affinity  and  do  not  exhibit  excessively  contrasting  properties. 
How  is  this  determined,  however? 

If  we  use  Paivio's  work  as  a  basis,  we  may  look  for  a  combination  of  associative  and 
referential  processing.  That  is,  we  could  look  for  auxiliary  metaphors  that  stimulate 
images  and  semantic  expressions  related  to  those  stimulated  by  the  underlying  metaphors 
they  are  intended  to  accompany.  One  may  think  of  the  underlying  metaphor  as  establishing 
the  general  context  into  which  the  auxiliary  metaphor  must  fit.  Properties,  operations, 
phrases,  images,  and  types  associated  with  the  underlying  metaphor  may  provide  the 
injaginal  and  semantic  links  to  related  metaphors  that  might  serve  as  auxiliary  metaphors. 
Thus,  one  might  begin  by  selecting  an  underlying  metaphor  and  then  begin  systematically  to 
identify  related  properties,  operations,  phrases,  images,  and  types  (POPIT).  Once  these 
were  identified,  in  addition  to  helping  to  determine  the  key  features  and  functions  of  the 
graphical  user  interface  (GUI)  based  on  that  underlying  visual  metaphor,  they  might  also 
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suggest  promising  auxiliary  metaphors. 


APPLYING  THE  POP1T  APPROACH 


What  does  this  approach  offer  us?  Let's  examine  two  frequently  used  underlying 
metaphors.  Applying  the  process  and  generating  ideas  in  each  of  the  five  areas  is  not 
necessarily  a  linear  process.  An  idea  generated  in  one  of  the  Ave  areas  may  stimulate 
ideas  in  other  areas.  This  tendency  to  %hop  around  mentally1  is  typical  of  brainstorming 
activities  (Osborn,  1963)  and  perhaps  offers  some  empirical  evidence  of  Paivio's  earlier 
discussed  theses  on  the  spread  of  activation. 


Many  hypermedia  programs  employ  the  book  metaphor.  That  is,  one  thinks  of  the  product 
as  a  book  and  each  screen  becomes  a  page.  The  GUI  might  well,  therefore,  portray  the 
screen  as  a  book,  perhaps  with  binding  on  one  side  and  pages  of  text  on  the  other  or  with  a 
page  of  text  on  the  left  side  of  the  screen  and  a  page  of  text  on  the  right  side  and  the  binding 
down  the  middle  of  the  screen  .  In  fact,  the  book  metaphor  is  the  one  employed  by 
Asymetrix's  ToolBookTM  hypermedia/multimedia  authoring  tool  to  help  authors  understand 
how  to  build  a  product.  Similarly,  the  graphics  needed  to  implement  the  book  metaphor  are 
supplied  by  Apple  with  their  authoring  tool,  HyperCardTM,  although  Apple  actually  uses 
index  card  (underlying)  and  stack  of  cards  (auxiliary)  metaphors  in  the  accompanying 
literature. 

First,  let's  consider  the  properties  of  a  book.  A  book  has  a  front  and  back  cover.  It  is 
bound.  It  is  printed  on  paper.  A  book  has  pages.  Pages  may  contain  text,  illustrations, 
photographs,  or  a  combination  of  the  three.  Pages  are  numbered.  Pages  may  be  organized 
into  chapters.  The  table  of  contents  comes  after  the  title  page  but  before  the  main  body  o 
text.  The  index  comes  after  the  main  body  of  text.  A  book  has  an  author  or  authors.  It  may 
have  an  illustrator.  Books  have  publishers.  Books  are  sometimes  published  in  later 
editions. 

Second,  what  operations  does  one  perform  in  using  a  book?  One  picks  up  a  book.  We 
may  take  it  from  a  bookcase  or  shelf  and  we  may  return  it  to  that  shelf  or  bookcase  when 
finished.  One  opens  a  book.  We  may  scan  a  page  or  we  may  read  it.  If  there  are 
illustrations  or  photographs,  we  may  examine  those.  We  can  reread  difficult  passages. 
One  turns  pages  forward  and  backwards.  One  may  use  a  table  of  contents  to  determine 
what  is  to  follow  and  an  index  to  help  locate  references  to  any  particular  phrase,  term,  or 
name.  One  can  look  ahead  in  a  book  or  look  back  to  re-examine  something  read  earlier. 
One  can  stop  reading  whenever  one  wishes.  One  can  mark  one's  place  with  a  bookmark. 
The  text  in  a  book  may  be  read  silently  or  aloud.  If  one  owns  the  book,  one  can  mark  in  it. 
One  can  copy  material  from  a  book,  either  by  hand  or  using  a  copy  machine.  Some  people 
moisten  their  fingertips  before  turning  a  page. 

Third,  let's  note  that  phrases  (or  terms)  associated  with  books  and  reading  include: 
binding,  pagination,  tome,  read  aloud,  read  silently,  turn  pages,  mark  your  place,  open, 
close,  begin  reading,  thumb  through  or  page  ahead,  quit  reading,  fold  the  corner  of  the  page 
down,  underline,  highlight,  write  marginal  notes,  margins,  and  read  with  your  finger. 

Fourth:  Images  associated  with  hooks  include  favorable  ones  like  leather  bound  books, 
illustrations,  onion  skin  paper,  and  the  smell  of  new  paper  in  new  books;  unfavorable  ones 
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like  dog-eared  or  smudged  pages,  the  smell  of  mildew  or  mold  in  old  books,  and  trails  from 
hungry  silverfish;  and  neutral  ones  like  hard  covers,  soft  covers,  hefty  books  and  paperback 
pocket  books. 

Lastly,  let's  identify  types  of  books.  Reference  works  include  dictionaries,  encyclopedias, 
atlases,  thesauruses,  almanacs,  and  the  like.  Recreational  books  include  the  paperback 
pocket  book,  the  mystery,  science  fiction  books,  fiction  and  non-fiction  books,  comic  books, 
magazines,  and  others.  Scholarly  books  include  textbooks,  the  Bible,  Torah,  Koran,  law 
books,  and  other  similar  works. 

How  do  these  five  areas  affect  how  we  implement  the  book  as  the  underlying  metaphor? 
The  hook  metaphor  carries  with  it  organization  and  sequence  and  a  variety  of  implied 
assumptions  about  how  one  uses  a  book,  and  thus,  any  product  'like  a  book.'  As  suggested 
earlier,  we  would  expect  the  product  to  use  pages  and  we  would  expect  to  be  able  to  "turn" 
pages    thnt  is,  to  move  forward  or  backward  by  pages.  We  might  well  expect  pages  to  be 
numbered  and  organized  into  chapters.  We  might  expect  to  see  a  table  of  contents  and  to 
he  able  to  use  (in  index  to  locate  particular  material.  Since  books  contain  primarily  words, 
illustrations,  and  photographs,  we  would  expect  to  see  these  in  a  product  using  a  book  as 
the  visual  metaphor  for  its  GUI.  We  might  well  expect  to  open  and  close  the  book.  We  would 
hope  to  he  able  to  Htop  whenever  we  wished  and  to  use  a  bookmark  to  mark  our  place.  Many 
reader*  might  like  to  be  able  to  mark  in  their  books.  If  so,  the  product  would  need  to  permit 
underlining  or  highlighting.  If  it  wished  to  allow  the  user  to  write  marginal  notes,  logically, 
it  must  firHt  have  margins.  To  avoid  a  "smudged"  look,  any  underlining,  highlighting,  or 
marginal  notes  would  need  to  appear  in  a  fashion  that  did  not  reduce  the  readability  of  the* 
page.  If  the  product  were  to  echo  associated  phrases,  perhaps  the  last  page  one  read  before 
stopping  would  have  its  page  corner  "folded  down"  until  one  started  reading  again,  at  which 
time  the  page  corner  would  "unfold,"  thus  avoiding  the  problem  of  dog-eared  pages. 
Similarly,  might  the  product  permit  thumbing  through  or  paging  ahead? 

What  auxiliary  metaphors  might  accompany  the  book  metaphor?  One  auxiliary  metaphor 
might  be  the  tab.  Some  books  -  usually  reference  works  (taking  our  cue  from  types)  --  have 
tabbed  sections.  These  tabs  permit  the  user  to  move  quickly  to  important  or 
frequently  used  sections  of  the  book.  In  many  books,  tabs  are  cut  into  the  pages  and  are 
rounded  to  accommodate  the  tip  of  the  human  finger.  Other  times,  the  tabs  project  beyond 
the  edge  of  the  page  so  that  they  act  as  dividers  separating  the  sections  of  a  book.  A 
product  using  the  tab  metaphor  would  be  expected  to  present  the  user  with  similar 
appearances  and  properties. 

This  suggests  a  second  auxiliary  metaphor  (this  one  derived  from  phrases):  the 
finger/thumb  metaphor.  Perhaps  the  product  could  employ  images  of  a  finger  in  some  way  as 
a  pointer,  a  place  marker,  or  an  indicator.  Perhaps  the  product  could  use  the  image  of  a 
thumb  in  some  way  to  indicate  the  ability  to  thumb  through  or  thumb  ahead.  The  auxiliary 
metaphor  here  is  one  of  making  the  interface  in  some  way  parallel  the  use  of  fingers  and 
thumbs  in  reading. 

A  third  auxiliary  metaphor,  this  time  based  on  operation        /nt  be  the  copy  machine. 
Might  the  product  in  some  way  facilitate  copying,  using  images  that  mirror  the  way  books  are 
actually  copied?  Could  the  operation  resemble  the  process  by  which  one  uses  a  copier, 
including  such  features  as  variable  number  of  copies,  enlargement  and  reduction  copying, 
collated  copies,  and  two-sided  copying  for  bound  copies?  Would  the  book  be  opened  faced 
down  on  top  of  the  copier?  Would  copies  come  out  one  side  of  the  copier? 
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Tlir  uientilicd  typi'H  of  hooks  offer  another  suggestion.  Perhaps  a  fourth  auxiliary 
metaphor  might  ho  the  overlay  metaphor.  In  encyclopedias  one  may  rind  a  si»t  of 
transparent  overlays  hound  into  the  hook.  Them.1  overlays  enable  the  reader  to  "build"  an 
image  out  of  several  layers  or  to  "uncover"  lower  or  internal  levels  by  removing  layers.  The 
most  frequently  encountered  example  of  sueh  a  eonstructed  image  is  probably  the  "visible* 
human  body:  One  layer  shows  the  skeleton;  one  shows  the  internal  organs;  and  one  shows 
the  skin.  We  would  expect  the  layers  to  "flip"  in  the  direction  of  the  binding.  In  most  cases, 
for  books  written  in  English,  we  would  expect  to  uncover  by  lifting  from  right  to  left  and  to 
build  by  placing  transparent  overlay  pages  from  the  left  on  top  of  similar  pages  on  the  right. 
If  the  binding  were  at  the  top,  we  would  expect  to  build  by  lowering  overlays  from  the  top  and 
to  uncover  by  lifting  pages  from  the  bottom.  A  product  employing  the  overlay  metaphor  would 
be  expected  to  have  some  pages  that  permitted  the  user  to  build  and  uncover  images  in 
keeping  with  this  metaphor. 

A  fifth  auxiliary  metaphor    also  drawn  from  types  »  might  be  the  pronunciation  key. 
Dictionaries  and  similar  reference  hooks  often  include  a  printed  pronunciation  key.  This  is  a 
set  of  letters  and  phonetic  representations  that  are  intended  to  help  the  user  figure  out  how 
to  pronounce  a  word  properly.  A  product  implementing  this  auxiliary  metaphor  might  provide 
the  user  with  the  ability  to  hear  the  word  pronounced.  This  is  clearly  the  first  auxiliary 
metaphor  that  goes  beyond  the  normal  capabilities  of  the  object  or  device  to  which  the  GUI  is 
being  compared.  Books  do  not  normally  speak  to  their  readers.  For  the  present,  however, 
let's  accept  this  implementation  as  valid.  We  will  consider  in  the  next  section  of  this  paper 
whether  this  is  a  legitimate  addition  or  a  violation  of  the  underlying  metaphor. 

A  sixth  auxiliary  metaphor  based  on  operations  is  the  shelf/bookcase  metaphor.  If  a 
hypermedia/multimedia  product  is  a  book,  then  it  can  be  part  of  a  shelf  of  books,  perhaps 
organized  by  topic,  time  period,  or  some  other  common  trait.  If  related  books  reside  on  the 
same  shelf,  then  perhaps  a  bookcase  houses  shelves  of  related  volumes.  Perhaps  one  could 
draw  books  from  different  shelves  as  needed.  Perhaps,  as  is  the  case  in  research  libraries, 
users  could  "read"  the  shelves  for  interesting  books,  confident  that  they  are  "in  the  right 
section"  because  related  books  are  shelved  together.  Among  other  functions,  this  auxiliary 
metaphor  might  be  used  to  help  organize  related  products  in  the  same  line. 

The  Map 

Another  visual  metaphor  for  a  hypermedia/multimedia  product's  interface  might  be  the 

map  .  Thus,  using  the  map  metaphor,  the  screen  becomes  a  map  of  a 

place. 

Properties  of  maps:  Maps  generally  focus  on  contiguous  areas  or  locations.  Maps  have 
legends  that  explain  the  symbols  they  use.  They  have  locations  that  are  spatially  related 
to  one  another.  Maps  may  represent  various  properties  of  the  places  they  represent. 
Individual  maps  often  fold.  Maps  are  often  hard  to  refold.  Many  maps  fold  out  to  be  large. 
Maps  are  often  complete  on  one  side  of  the  page.  Some  maps,  particularly  those  in  atlases  or 
book  form,  will  have  notes  at  the  edges  of  individual  maps  to  inform  the  user  of  where  that 
map  connects  with  a  related  map.  Maps  often  use  very  small  print.  Many  maps  use  color  to 
help  users  distinguish  features  and  relations.  Maps,  particularly  those  that  depict  routes, 
usually  provide  an  index  of  street  or  location  names.  Route  maps  often  supply  tables  of 
distance  or  travel  time  as  aids  to  users.  Some  maps  include  enlarged  views  (insets)  of 
important  areas,  such  as  cities. 
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Map  operations  include  unfolding  and  refolding,  reading  the  legend,  getting  the  map 
oriented  in  terms  of  what  you're  looking  for,  marking  your  route,  and  noting  interesting 
locations  along  a  route. 


Phrases  include  getting  one's  bearings,  figuring  out  which  way  is  up,  reading  the  map, 
finding  the  best  route,  plotting  a  course,  getting  the  lay  of  the  land,  calculating  distances  or 
times,  no  one  co  \\d  refold  this  thing,  men  never  use  maps  or  stop  to  ask  directions  when  lost, 
V  marks  the  spot,  you  are  here,  and  putting  your  finger  right  on  it. 

Among  the  images  that  come  to  mind  are  maps  covering  table  tops;  the  difficulty  of 
reading  a  map  in  a  constricted  space  like  a  car,  the  difficulty  of  reading  the  small  print;  the 
difficulty  of  refolding  a  map  without  creating  an  awkward  bundle;  and  the  colorful  and 
pleasing  appearance  of  well  produced  maps. 

Types  of  maps  include  topographical  and  geophysical  maps,  geosocial  and  geopolitical 
maps,  directional  or  route  maps,  maps  that  depict  demographics  -  characteristics  of  the 
people  who  live  in  the  places  depicted  -  and  maps  that  describe  properties  of  the  region 
represented,  such  as  economic  maps.  Maps  may  be  presented  individually  or  as  a  set 
(atlas). 

What  are  the  implications  of  our  POPIT  analysis?  We  would  expect  any 
hypermedia/multimedia  product  that  used  a  map  as  its  underlying  metaphor  to  use  the 
map  to  depict  snatially  related  locations.  It  should  include  a  legend  to  explain  any  symbols 
involved  and  should  indicate  which  way  is  up  (phrases).  Color  might  serve  to  help  define 
relationships.  Products  using  the  map  metaphor  might  offer  indexes  to  street  or  place  names 
or  tables  of  distance  or  time. 

From  operations:  An  accompanying  auxiliary  metaphor  here  might  be  the  inset.  A 
product  that  implemented  the  inset  metaphor  might  be  expected  to  include  enlarged  views 
on  screen.  Another  option  here,  drawn  from  properties  and  images,  might  be  to  think 
about  how  users  obtain  enlargements  when  none  is  provided:  the  magnifying  glass. 
Perhaps  the  magnifying  glass  metaphor  could  serve  as  an  alternative  auxiliary  metaphor  to 
the  inset.  The  product  would  then  make  available  to  the  user  a  magnifying  glass  that,  when 
passed  over  the  surface  of  the  map,  revealed  greater  detail.  In  a  typical  mouse-  directed 
GUI,  one  might  expect  to  see  the  traditional  pointed  arrow  mouse  pointer  replaced  by  a 
magnifying  glass.  The  images  on  the  map  would  then  become  enlarged  as  the  user  moved 
the  mouse  and  its  correlated  magnifying  glass  over  the  surface  of  the  map  on  the  screen.  In 
the  case  of  the  magnifying  glass,  as  in  the  case  of  the  pronunciation  key,  we  have  shifted 
slightly  from  the  original  device  (the  inset).  We  have  not  changed  the  intention.  What  we 
have  changetd,  however,  is  how  that  intention  is  accomplished. 

From  properties:  Since  individual  maps  fold  and  unfold,  products  could  use  the  folded 
map  auxiliary  metaphor  to  make  it  possible  to  have  access  to  many  maps  from  the  same 
screen.  Maps  could  be  unfolded  and  refolded.  Noting  the  difficulty  many  people  have  in 
refolding  maps  and  the  sense  of  frustration  that  act  can  produce  (see  properties,  phrases, 
and  images  above),  the  product  could  attempt  to  automate  the  process  in  a  visually  -  and 
perhaps  auditorially  -  satisfying  way. 

From  types:  A  collection  of  products  based  on  the  map  underlying  metaphor  might  be 
clustered  together  using  the  auxiliary  metaphor  of  atlas.  Like  the  shelf/bookcase 
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metaphor,  the  atlas  metaphor  makes  it  possible  to  organize  an  entire  line  of  products  in  a 
way  that  makes  clear  to  the  user  their  relationships  to  one  another.  Taking  a  cue  from 
properties,  these  relationships  might  be  clarified  further  by  notations  on  the  edges  of 
maps  stating  which  other  maps  connect  to  those  edges. 

EVALUATING  MKTAPHORIC  INTERACTION:  COMPLEMENTARY  OR  CONFOUNDING 

We  have  attempted  above  to  derive  a  network  of  related  or  associated  elements  that  help  to 
define  and  describe  an  underlying  metaphor.  We  have  then  used  those  elements  to  help  us 
identify  how  we  would  expect  a  hypermedia/multimedia  product  employing  that  underlying 
metaphor  to  operate.  In  addition,  we  have  examined  a  number  of  possible  auxiliary 
metaphors  one  might  use  in  a  product  based  on  a  specific  underlying  metaphor.  In  Kittay's 
(1987)  terms,  we  have  attempted  to  use  an  analysis  of  these  elements  to  help  us  identify 
implementations  of  the  metaphors  that  exhibit  high  degrees  of  affinity  and  acceptable  levels 
of  contrast.  Stated  in  simpler  term*,  our  goal  is  to  implement  visual  metaphors  that  operate 
in  ways  that  are  consonant  with  the  experiences  of  our  users  in  working  with  the  object  or 
device  to  which  we  are  comparing  our  product's  interface  (our  underlying  metaphor).  Thus,  a 
product  based  on  the  book  metaphor  should  (1)  bear  some  visual  resemblance  to  a  book  and 
(2)  operate  in  ways  very  similar  to  (consonant  with)  how  a  Veal'  book  operates.  Similarly, 
auxiliary  metaphors  should  complement  -  add  to,  enhance,  help  make  complete  the 
underlying  metaphor.  When  contrast  occurs,  it  should  be  of  a  type  that  encourages  users  to 
reinterpret  and  reconstruct  (Miller,  1979)  their  understanding  how  the  interface  operates.  If 
users  finds  that  the  contrast  is  too  great,  however,  one  might  expect  them  to  reject  the 
comparison.  When  this  occurs,  we  may  term  the  interaction  between  the  underlying  and 
auxiliary  metaphors  as  confounding.  The 

interaction  between  underlying  and  auxiliary  metaphors  is,  therefore,  either  complementary 
or  confounding. 

Since  our  earlier  examples  have  all  attempted  to  be  complementary  implementations,  this 
seems  a  logical  time  to  examine  some  implementations  that  seem  confounding.  Consider  a 
product  having  as  its  underlying  metaphor  the  book.  An  auxiliary  metaphor  of  a  videotape 
player  seems  clearly  confounding,  A  book  may  not  contain  a  videotape  player.  When  users 
are  faced  with  such  an  auxiliary  metaphor  they  are  required  to  reconstruct  the  environment 
radically,  envisioning  a  book  that  is  unlike  any  that  the  user  has  ever  seen.  Users  seem 
unlikely  to  make  such  radical  reconstructions.  Instead,  what  they  will  probably  do  is  to 
decide  that  some  parts  of  the  product  act  'something  like  a  book'  and  others  act  like  other 
things.  When  users  come  to  this  conclusion,  the  benefits  of  the  underlying  metaphor  are 
greatly  reduced.  The  dynamic  tension  inherent  in  the  metaphor  (Kittay,  1987)  is  removed; 
the  user  no  longer  feels  compelled  to  interpret  and  reconstruct  to  make  the  metaphor  fit. 

Consider  a  product  that  employs  an  underlying  metaphor  of  the  map.  While  the  earlier 
discussed  auxiliary  metaphor  of  folding  seems  complementary,  an  auxiliary  metaphor  of  a 
toolbox  seems  confounding.  We  would  not  expect  maps  to  contain  toolboxes.  No 
reconstruction  that  the  user  may  perform  can  create  a  map  capable  of  offering  boxes  of 
tools.  Instead,  the  user  will  most  likely  decide1  that  Home  parts  of  the  interface  operate  'like  a 
map*  and  other  parts  operate  in  some  completely  different  way. 

Earlier  in  this  paper  we  considered  two  auxiliary  metaphors  that  seemed  perilously  close  to 
confounding.  The  first,  the  pronunciation  key,  offered  users  a  chance  to  hear  a  word 
pronounced  when  they  clicked  on  it.  The  second,  the  magnifying  glass,  offered  users  the 
chance  to  pass  a  magnifying  glass  over  a  portion  of  the  screen  in  order  to  see  an 
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enlargement  of  what  appeared  on  the  screen.  Why  are  these  two  metaphors  not 
confounding?  Let's  examine  them. 

It  is  worth  noting  that  there  are  published  books  -  mostly  for  children    that  do  offer  the 
reader  the  chance  to  hear  recorded  sound  played  back.  In  fact,  one  can  buy  greeting  cards 
that  piny  tunes  or  messages  and  picture  frames  that,  in  the  words  of  their  marketers,  "allow 
your  loved  ones  to  hear  your  recorded  message  each  time  they  touch  your  picture.11  Thus, 
users  may  well  be  able  to  reconstruct  their  interpretation  of  the  product  as  a  book  that  -  l"ke 
those  children's  books  and  the  greeting  cards  -  is  capable  of  allowing  them  to  hear  a  recorded 
sound.  Another  reason  that  this  metaphor  may  work  is  that  readers  using  a  printed 
pronunciation  key  in  a  book  usually  attempt  to  use  that  guide  to  pronounce  the  word  aloud 
as  they  read  it.  Users  may,  therefore,  be  accustomed  to  hearing  the  pronunciation  as  they 
read  the  key.  While  the  product's  implementation  changes  the  source  of  that  aural  stimulus, 
it  does  not  seem  excessively  dissonant  with  users'  own  experiences  (Brown,  1988). 

It  is  also  well  within  most  users'  experiences  to  use  a  magnifying  glass  to  read  printed 
materials  that  are  small.  In  addition,  when  reading  a  map,  one  might  well  lay  the 
magnifying  glass  down  on  the  map  outside  the  area  one  was  currently  examining  and  then 
lift  it  and  pass  it  over  any  area  one  wished  to  see  enlarged.  Thus,  although  maps  do  not 
contain  or  offer  magnifying  glasses,  the  use  of  the  one  with  the  other  can  easily  be 
reconciled,  and  the  user  can  reconstruct  the  metaphorical  comparison  in  consonant  vays 
that  enhance  the  interaction. 

How  then  do  we  handle  our  need  to  include  such  functions  as  video  segments  and  tools  in 
hypermedia/multimedia  products?  One  way  would  be  to  select  as  underlying  metaphors 
ones  that  permit  us  to  include  a  wider  range  of  auxiliary  metaphors.  While  a  book  cannot 
offer  a  videotape  player,  many  settings  could  offer  such  a  device.  For  example,  it  would  not 
be  unreasonable  to  find  a  videotape  player  in  an  entertainment  center  or  production  studio. 
It  also  would  not  be  unusual  to  find  books  in  such  settings.  By  selecting  a  broader 
underlying  metaphor,  one  can  include  more  auxiliary  metaphors.  A  second  way  to  include 
needed  functions  might  be  to  consider  ways  in  which  the  subject  of  the  underlying  metaphor 
might  offer  similar  functions.  For  example,  an  electronic  journal  might  be  handled  in  the 
form  of  marginal  notes  or  notes  written  on  a  separate  pad  while  reading.  A  third  possible 
solution  might  be  to  think  of  the  metaphor  less  in  terms  of  places  and  things  and  more  in 
terms  of  events  or  chains  of  events.  That  is,  one  could  use  a  metaphor  that  involves  a 
sequence  of  events,  like  traveling,  where  location  and  function  change  in  predictable  ways 
that  invite  the  use  of  multiple  auxiliary  metaphors.  Thus,  in  traveling,  one  might  get  tickets 
or  make  reservations,  board  the  method  of  transportation  (embark),  travel  on  board  with 
whatever  events  occur  thereon,  arrive  at  one's  destination  disembark),  have  experiences  at 
the  new  location,  re-board  transportation,  travel  back,  and  disembark  at  the  original 
location.  Since  the  concept  of  traveling  is  predictable  and  yet  varied,  traveling  may  prove  a 
useful  underlying  metaphor.  Traveling  on  a  plane  offers  its  own  set  own  events  that  differ 
from  the  events  available  on  a  cruise  or  in  a  car.  It  is  not  difficult  to  think  of  other  similar 
sequential  metaphors.  One  that  come  quickly  to  mind  include  chronology  (particularly  in 
terms  of  birth  to  death),  production  (perhaps  from  raw  material  to  finished  product),  and 
construction  (for  instance,  foundation  to  roof)  or  its  inverse,  demolition. 

METAPHORS  RECONSIDERED 

By  now,  it  should  he  clear  why  thiH  paper's  title  compares  the  process  of  selecting 
appropriate  metaphors  to  the  damnation  of  the  underworld.  The  process  is  neither 
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uncomplicated  nor  free  of  risk.  It  is  difficult  to  decide  how  much  or  how  little  metaphor  to 
use,  While  an  interface  devoid  of  metaphor  may  require  more  effort  from  its  users  because  it 
does  not  permit  them  to  use  other,  non-computer  experiences  to  assist  in  understanding  and 
predicting  the  actions  of  the  interface,  an  interface  encumbered  with  too  many  metaphors 
may  overwhelm  users,  leaving  them  confused,  irritated,  and  worse  off  than  in  a  many  chunks 
of  information  oik*  can  handle  und  Furukawa's  (1970)  research  on  cognitive  load,  designers 
may  wish  to  choose  one  underlying  metaphor  and  no  more  than  five  to  eight  auxiliary 
metaphors.  The  more  closely  tlnve  auxiliary  metaphors  are  related  to  the  underlying 
metaphor,  the  less  additional  rngiiiMvi1  load  their  presence  should  place  on  the  user  (Andre  & 
Phye,  19S6;  Oren,  1990).  That  is,  the  less  difficult  it  should  be  for  the  user  to  recognize  how 
they  relate  to  the  underlying  metaphor  and  to  one  another  and  the  easier  it  should  be  for  the 
user  to  incorporate  them  into  a  unified  Interpretation  of  the  interface. 

Not  everyone  supports  the  use  of  metaphor  in  hypermedia/multimedia  products.  For 
example,  Nelson  (1990)  argued  thai  dependence  on  the  use  of  metaphors  for  all  aspects  of  a 
multimedia  system  could  kill  a  product,  Kxtensive  use  of  metaphor  is  undesirable, 
according  to  Nelson,  because  thu  relationship  between  the  product  and  the  object  or  device 
to  which  it  is  being  compared  is  usually  tenuous  at  best  and,  once  a  metaphor  is 
implemented,  everything  has  to  become  a  part  of  it.  He  suggested  instead  that  designers 
identify  "well-thought-out  unifying  ideas,  embodied  in  richer  graphic  expressions  thac  are  not 
chained  to  silly  comparisons"  (p.  237). 

While  anyone  who  has  had  the  opportunity  to  observe  a  large  enough  sample  of 
hypermedia/multimedia  products  can  attest  to  the  existence  of  numerous  examples  of  just 
such  misuse  and  overuse  of  metaphor,  this  fact  may  be  more  reflective  of  inadequate 
analysis  of  the  metaphors  employed  and  lack  of  attention  to  the  relationship  between 
underlying  and  auxiliary  metaphors  than  of  the  weaknesses  of  metaphorical  thinking  itself. 
Nelson's  argument  that  we  should  use  'richer  graphic  expressions'  seems  reasonable,  but 
those  graphic-expressions  are  themselves  rooted  in  metaphorical  thinking.  Few  things  on  a 
computer  screen  are  what  they  appear  to  be,  particularly  in  a  GUI.  As  Langer  (1958)  noted, 
a  label  is  not  what  it  describes*,  nor,  as  Magritte  pointed  out,  is  a  picture  that  which  it 
represents.  Recognizing  this,  an  earlier  work  proposed  a  model  for  considering  the  properties 
of  the  graphics  used  as  icons  (Cates,  1993).  It  seems  unlikely  that  we  can  -  or  should  - 
avoid  metaphors  in  hypermedia/multimedia  products.  As  Lakoff  and  Johnson  (1980) 
suggested,  metaphorical  thinking  is  so  pervasive  in  our  society  that  we  cannot  avoid  it.  If 
dual  coding  plays  a  major  role  in  the  operation  of  our  cognitive  processes  (Paivio,  1986), 
users  cannot  easily  separate  the  image  from  the  related  verbal  labe \  nor  can  they  isolate 
themselves  from  the  associated  spread  of  activation  that  occurs  naturally  when  they  are 
exposed  to  either  verbal  labels  or  images. 

Laurel  (1990)  suggested  that,  "People  transform  their  interfaces.  Interfaces  also  transform 
their  users"  (p.  91).  As  designers,  we  face  two  choices:  (1)  V/e  can  ignore  the  spread  of 
activation  in  the  users  of  our  products  and  pretend  that  interfaces  are  not  affected  by  this 
spread  of  activation,  or  (2)  We  can  do  our  very  best  to  analyze  metaphors,  predict  the  likely 
types  of  activations,  and  consider  the  ways  in  which  users  interact  with  our  interfaces.  The 
latter  choice  seems  the  only  reasonable  one.  Making  this  choice  calls  for  a  commitment, 
however.  In  order  to  produce  the  most  effective  interfaces,  we  will  need  to  analyze  our 
metaphors  carefully  before  we  decide  to  include  them  in  our  products  and  we  will  need  to 
collect  and  analyze  data  on  how  users  interact  generally  with  our  products  and  specifically 
with  the  metaphors  those  products  contain. 
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INTRODUCTION 


The  relationships  among  graphical,  motivational,  and  learning  variables  are  complex  in 
any  instructional  method,  but  especially  in  interactive  learning  environments.  Whereas 
media  such  as  textbooks  and  television  basically  provide  graphics  or  video  images  for 
passive  viewing,  computer-based  media  incorporate  graphics  that  engage  learners  in  more 
dynamic  ways  (Nishida,  1992;  Alesandrini,  1987).  The  two  (and  sometimes  three) 
dimensional  integration  of  graphics  with  text  as  an  integrated  whole  in  interactive  learning 
environments  has  the  potential  to  motivate  students  to  interact  more  directly  and 
voluntarily  with  graphicul  information.  This  potential  is  greater  than  in  printed  materials 
in  which  graphics  often  play  a  more  ancillary  role. 

Some  maintain  that  computer  graphics  design  has  entered  a  new  era  with  the  advent  of 
3-D  animated  graphic**  and  virtual  reality  systems  (Wooley,  1992).  The  principles 
underlying  the  role  of  graphics  in  motivating  students  and  supporting  their  knowledge  and 
skill  acquisition  must  be  incorporated  into  the  design  of  all  types  of  interactive  learning 
environments,  especially  if  these  environments  are  expected  to  support  higher-order  learning 
such  as  the  construction  of  mental  models  (Friedman,  1993).  If  the  hypothesis  that  griipliicH 
play  an  important  role  in  helping  students  to  acquire  knowledge  and  skills  and/or  in 
increasing  students'  motivation  to  persevere  in  computer-based  learning,  then  guidelines  for 
the  design  of  interactive  learning  environments  should  include  prescriptions  for  the  use  of 
graphics  that  maximize  motivation  and  learning. 

There  is  a  body  of  literature  that  addresses  design  guidelines  for  integrating  graphics 
into  computer-based  instructional  materials  (e.g.,  Alesandrini,  1985;  Alesandrini,  1W87; 
Faiola  &  DeBloois,  1988;  Marcus,  1992;  Morrison,  Ross,  O'Dell,  Schultz,  &  Higginbothniu- 
Wheat,  1989).  However,  the  guidelines  are  mostly  concerned  with  how  the  technical  as):  jets 
of  graphics  (e.g.,  color  or  realism)  can  be  integrated  into  the  design  of  computer  based 
instruction.  Although  the  integration  of  technical  innovations  (more  colors,  more  pixels, 
faster  access  times)  provides  a  wide  variety  of  new  opportunities  for  instructional  deaignern 
to  dazzle  their  audiences,  the  justification  for  using  specific  graphic  representations  and 
treatments  should  be  based  upon  a  "learner-centered"  perspective,  rather  than  a 
"technocentric"  one  (Rieber,  1989a).  In  other  words,  instead  of  evaluating  interactive 
materials  v>ased  on  how  well  the  capabilities  of  the  hardware  and  software  are  utilized,  it  is 
much  more  important  to  assess  the  roles  graphics  play  as  instructional  dimensions  that 
enhance  learning  and  motivation. 

Despite  the  widespread  use  of  graphics  in  instruction,  the  psychological  literature  about 
"how"  people  acquire  knowledge  and  make  inferences  from  graphics  is  sparse.  How  learners 
integrate  text  and  graphics  and  whether  their  ability  to  learn  from  graphics  varies  with  their 
level  of  visual  literacy,  experience,  or  expertise  within  a  specialized  domain  are  also 
unresolved  questions  (Friedman,  1993).  There  is  s^me  evidence  that  people  with  different 
experience  in  a  specific  knowledge  area  employ  different  strategies  in  viewing  and  interacting 
with  graphics  (ChanLin,  1993;  Horton,  1991).  For  example,  given  specific  instructional 
objectives,  the  use  of  abstract  graphics  to  present  scientific  information  to  more-experienced 
viewers  might  be  contra  die  tea  with  less-experienced  viewers.  Unfortunately,  there  are  few 
empirically-based  guidelines  about  how  best  to  convey  a  specific  graphic  image  to  learners 
that  have  different  levels  of  familiarity  with  the  content  of  the  graphic.  Instead,  research  in 
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graphics  in  interactive  learning  environments  (and  other  media  for  education  and  training) 
has  focused  on  the  physical  variables  of  graphics,  such  as  pictures  vs.  drawings  or  color  vs. 
monochrome  or  large  vs.  small  displays  (cf.,  Friedman,  1993).  What  we  really  want  to  know 
is  how  graphics  enhance  .Motivation  and  support  learning  across  many  different  individual 
characteristics,  objectives,  media,  and  other  instructional  dimensions. 

The  main  objective  of  this  paper  is  to  review  research  that  has  been  conducted  related  to 
the  use  of  graphics  and  how  various  design  variables  interact  wi;h  graphics  in  influencing 
the  process  of  learning  within  individuals.  It  is  also  the  intent  of  this  paper  to  describe 
research  directions  and  approaches  that  may  provide  enhanced  guidelines  for  integrating 
graphics  into  future  interactive  learning  environments. 

PREVIOUS  RESEARCH  ON  GRAPHICS 

In  the  past,  graphics  research  focused  extensively  on  the  effects  of  graphics  in  various 
media  (e.g.,  in  books  versus  computer-based  instruction)  or  in  various  formats  (e.g.,  static 
versus  animated).  The  primary  methodologies  employed  in  these  studies  involved 
comparing  different  types  of  instruction  with  different  graphical  treatments.  The  findings  of 
this  research  are  not  always  in  favor  to  the  use  of  graphics  or  specific  graphical  formats.  For 
example,  a  specific  graphic  format  might  be  found  to  facilitate  memorization  among  a 
specific  group  of  learners  engaged  in  a  specific  recognition  task,  but  fail  to  achieve  the  same 
instructional  intent  among  another  group  of  learners  or  with  a  different  task  (Dwyer.  1987). 
Although  it  is  dismaying  that  so  many  of  the  graphical  design  guidelines  have  been  issued  in 
the  absence  of  empirical  evidence  (Friedman,  1993),  this  tendency  simply  reflects  the 
complexity  of  establishing  the  effects  of  graphics  in  interactive  learning  environments.  The 
situation  is  further  complicated  by  the  fact  that  what  limited  research  base  exists  was 
established  in  relatively  simple  computer-based  displays  (e.g.,  forty  characters,  line 
drawings,  and  eight  colors  on  one  screen)  rather  than  in  the  multi-windowed,  photo-realistic, 
full-spectrum  computer  presentations  that  are  the  hallmark  of  contemporary  interactive 
learning  environments. 

Following  precedents  set  by  traditional  graphic  researchers,  many  recent  studies  have 
tested  hypotheses  using  interactive  systems  that  were  already  examined  in  text  and  other 
media.  Each  of  these  studies  has  taken  a  slightly  different  perspective  on  how  to  improve 
learning  from  graphics.  For  example,  some  studies  focused  on  comparing  the  relative 
learning  effectiveness  between  computer-based  instruction  (CBI)  with  graphics  and  CBI 
without  graphics  (e.g.,  Alesandrini  &  Rigney,  1981).  Other  studies  were  carried  out  to 
examine  the  relative  effectiveness  of  different  degrees  of  realism  in  computer  graphics  (e.g., 
Gage,  1989).  Still  others  compared  the  use  of  different  graphic  cues  in  guiding  students' 
attention  (e.g.,  Beck,  1987).  These  studies  tried  to  support  the  assumptions  and  beliefs  of 
proponents  of  CBI  about  the  role  of  graphics  in  interactive  learning.  Example  beliefs  include: 
1)  pictures  are  effective  because  they  are  easier  to  perceive  than  verbal  information;  2) 
learning  is  somehow  proportional  to  degree  of  fidelity  in  graphics;  and  3)  pictures  provide 
redundant  codes  for  processing  information. 

Some  of  these  studies  have  provided  interesting  results.  For  example,  several  studies 
have  been  conducted  to  compare  graphical  versus  verbal  instruction  in  guiding 
comprehension  of  science  content  within  different  scientific  fields.  Rigney  and  Lutz  (197fi) 
compared  two  versions  of  chemistry  CAI  tutorials,  an  all-verbal  presentation  and  a  verbal- 
plus-graphics  presentation.  The  results  showed  that  learners  who  studied  the  verbal-plus- 
graphics  version  scored  higher  on  tests  of  knowledge,  comprehension,  and  application  of 
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concepts.  More  favorable*  ratings  of  the  learning  materials  were  also  obtained  from  the 
verbal'plus*graphics  group. 

To  test  tin*  effects  of  graphics  in  CBI,  two  experimental  studies  were  conducted  by 
Alcsandrini  arid  Kidney  (1981)  in  introductory  psychology  courses.  In  the  first  experiment 
students  were  assigned  to  cither  all  verbal  or  verbal-graphic  conditions;  learning  results 
favored  the  verbal-graphics  presentation.  The  second  experiment  was  conducted  to  test  the 
effectiveness  of  u  pictorial  review  task  compared  to  a  read-twice  control.  The  results 
indicated  that  the  pictorial  review  condition  facilitated  performance  more  than  re-reading; 
pictorial  review  produced  more  favorable  attitudes  as  well.  If  these  studies  were  exemplary 
of  most  of  the  extant  research,  the  hypothesis  that  pictures  provide  beneficial  effects  in 
terms  of  guiding  and  reinforcing  learning  in  CAI  would  be  strongly  supported. 

Unfortunately,  the  support  for  using  graphics  in  interactive  instructional  materials  is 
inconsistent.  In  a  reading  comprehension  study,  Edyburn  (1982)  found  no  significant 
increase  in  performance  for  a  graphic  CAI  group  compared  with  a  non-graphic  CAI  group.  In 
learning  comma  usage,  the  results  of  Smith's  (1985)  study  showed  that  a  graphic  treatment 
was  not  significantly  better  than  a  non-graphic  treatment.  Surber  and  Leeder  (1988) 
reported  that  provision  of  graphic  feedback  in  CAI  materials  did  not  enhance  either 
achievement  or  motivation  in  spelling.  One  explanation  for  the  inconsistent  results  among 
these  studies  may  be  that  related  variables  such  as  variation  in  learners'  abilities, 
differences  in  the  nature  of  the  learning  tasks,  and  variance  in  design  quality  across 
different  CAI  programs  have  not  been  sufficiently  assessed  or  controlled  in  the  studies.  In 
short,  the  relationship  between  graphics  and  learning  is  probably  much  more  complex  than 
the  relatively  simplistic  research  designs  employed  to  date  can  handle. 

Whereas  some  designers  of  interactive  learning  environments  seem  to  believe  that 
graphics  should  be  as  realistic  as  possible  to  enhance  motivation  and  learning,  the  degree  of 
realism  required  in  graphics  may  vary  for  different  instructional  tasks  (Dwyer,  1970).  Gage 
(1989),  for  example,  used  three  different  formats  fur  computer  graphics  among  three 
treatment  groups.  The  three  treatment  groups  used  highly  realistic  graphite,  moderately 
realistic  graphics,  and  slightly  realistic  graphics.  A  control  group  received  text  instruction 
with  illustrations.  The  analysis  of  posttcst  scores  with  the  pretest  as  a  eovnrinte  showed 
significant  differences  between  treatments.  The  results  showed  that  the  treatment  with  the 
least  realistic  graphics  had  the  highest  achievement.  Highly  detailed  graphics  did  not 
contribute  to  learning  in  this  study. 

Interpreting  this  study  requires  the  researcher  to  consider  cognitive  and  metacognitive 
process  rather  than  simple  perception  variables.  Perhaps  the  main  function  of  graphics  in 
some  contexts  is  to  help  students  learn  by  differentiating  relevant  cues.  Therefore,  excessive 
realism  may  actually  interfere  with  the  transmission  of  information  because  irrelevant  cues 
might  also  be  activated  in  processing  the  information  from  realistic  images.  From  the 
standpoint  of  economy  and  instructional  effectiveness,  simpler  graphics  may  interfere  less 
and  thus  be  more  effective.  Dwyer  concluded  that  different  graphical  formats  assist  learning 
and  comprehension  differently  (see  reviews  by  Dwyer,  1970,  1988)>  According  to  Dwyer,  if 
the  purpose  of  instruction  is  to  bring  the  students  into  close  touch  with  reality,  e.g.,  to 
encourage  an  emotional  response,  a  realistic  graphical  format  may  be  more  effective  than  an 
abstract  one.  The  appropriateness  of  realistic  details  in  graphics  is  related  to  differences  in 
the  learning  tasks  given  to  the  student,  the  time  allowed  for  interacting  with  graphics,  and 
the  cognitive  capabilities  of  students  to  process  the  information  provided  in  the  graphics. 
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GRAPHICAL,  MOTIVATIONAL,  AND  LEARNING  VARIABLES 

Although  the  integration  of  new  design  elements  (e.g.,  QuickTime  movies*)  is  a  popular 
trend  in  producing  graphics  for  interactive  learning,  the  theoretical  anil  cmpi.!.„|] 
foundations  for  their  use  have  not  been  established  by  research.  It  appears  that  many 
dimensions  of  graphics  themselves  are  poorly  understood  with  respect  to  their  instructional 
effects  and  implications  for  design.  Which  format  of  graphics  to  use,  when  to  use  it,  how  to 
use  it,  what  to  use  it  with,  and  to  whom  to  apply  it  are  all  reasonable  questions  that 
suggest  the  complexity  of  the  relationships  among  graphics,  motivation,  and  learning. 

Some  researchers  and  theorists  have  attempted  to  deal  with  this  complexity.  For 
example,  Kaufmann  (1985)  found  that  the  effects  of  graphics  vary  with  task  familiarity  and 
problem-solving  mode,  with  imagery  being  particularly  useful  in  novel  situations.  Other 
studies  have  examined  the  effects  of  different  elaboration  of  visual  materials  in  different 
levels  of  practice  in  computer-based  learning  materials  (e.g.,  Rieber,  1989b;  Rieber,  1990a, 
b;  Rieber,  Boyce,  &  Assad,  1990;  Rieber  &  Hannafin,  1988).  Researchers  also  investigated 
the  affective  effects  of  graphics,  such  as  in  meeting  emotional  needs,  as  an  essential  element 
in  supporting  motivation  and  knowledge  acquisition  (ChanLin,  1993;  Eisner,  1985).  These 
studies  assume  that  the  relationships  among  graphics,  motivation,  and  learning  should  not 
he  studied  unless  both  the  characteristics  of  learners  and  the  learning  environment  are  well 
integrated  into  the  research. 

Task  differences  may  also  piny  an  important  role  in  explaining  the  effects  of  graphics, 
('tmmugliam's  (l!WH)  meta-analysis  concluded  that  the  most  effective  subject  areas  for 
employing  computer  graphics  in  learning  materials  are:  geometry,  physics,  chemistry,  and 
health  education   Cuiininghiim  reported  that  representational  graphics  were  the  most 
widely  used  graphic  level  in  these  subjects,  while  animated  gruphics  were  the  most 
efficiently  employed  graphic  form. 

Different  tasks  and  different  access  conditions  for  interacting  with  pictures  may  also 
influence  the  level  of  understanding  learners  achieve  from  graphics,  Thomas  (1989)  studied 
computer  graphics  integrated  into  a  problem-solving  activity.  He  found  that  task  differences 
influenced  the  level  of  interaction  with  graphics.  Computer  graphics  integrated  into  problem- 
solving  activities  improved  students'  attitudes  and  achievement  in  learning  functional 
concepts,  but  not  in  learning  transformational  concepts.  In  addition,  compared  with  a 
divergent  approach  (students  allowed  to  change  graphics  throughout  the  activities),  the 
convergent-task  approach  (select  a  picture  to  work  with  throughout  the  activities)  resulted  in 
higher  levels  of  task-motivation  and  helped  students  develop  a  better  understanding  of 
functions. 

Computer  graphics  may  also  be  employed  as  a  learning  aid  in  terms  of  a  memory 
organizer  for  formation  of  concepts.  For  example,  Krahn  and  Blanchaer  (1986)  use 
computer- generated  graphics  as  organizers  to  help  students  learn  medical  concepts. 
Steinberg,  Baskin  and  Hofer  (1986)  and  Steinberg,  Baskin  and  Matthews  (1985)  used 
computer-generated  graphics  as  a  memory  tool  for  students  to  employ  in  problem-solving 
tasks.  Presenting  key  information  in  instruction  together  with  visuals  can  serve  as  a 
memory  cue  for  encoding  and  retaining  the  knowledge  concepts. 

There  is  also  evidence  indicating  that  graphics  might  have  differential  effectiveness  with 
different  types  of  learners.  In  a  study  of  the  effects  of  computer  graphics  in  first  year  algebra 
CBI,  Payne  (1988)  investigated  the  effects  of  instruction  on  solving  equations  graphically  to 
instruction  on  solving  linear  equation  algebraically,  There  were  no  significant  differences 
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between  experimental  subjects  (using  computer  generated  uruphH  prior  to  traditional 
instruction)  and  control  subjects  (receiving  traditional  instruction  only).  However,  students 
with  high  general  reasoning  or  computational  learning  scores  who  hail  the  graphical 
treatment  achieved  more  on  higher  level  behaviors  than  their  counterparts  who  experienced 
the  algebraic  treatment. 

The  use  of  different  instructional  strategies  may  also  influence  the  degree  to  which 
students  can  benefit  from  specific  types  of  graphics.  To  evaluate  various  strategies  used 
together  with  graphics  in  an  algebra  word  problem-solving  program,  four  experiments  were 
conducted  among  undergraduate  students  (Reed,  1985).  It  was  observed  that  the  effective 
use  of  graphics  requires  a  "learning  by  coaching"  condition,  especially  for  low  achievement 
students.  From  Reed's  (1985)  and  Thomas1  (1989)  studies,  it  is  suggested  that  the 
instructional  strategies  employed  for  presenting  graphics  may  influence  the  way  information 
is  processed  and  encoded. 

Although  using  pictures  might  promote  deeper  levels  of  mental  processing  during 
encoding,  their  presence  may  provide  little  facilitation  of  memory  tasks  in  other  contexts. 
Rieber  and  his  colleagues  examined  the  effects  of  computer  animated  graphics  in  physics 
instruction  among  different  grade  levels  (Rieber,  1989b;  Rieber,  1990a,  b;  Rieber,  Boyce  & 
Assad,  1990;  Rieber  &  Hannafin,  1988).  They  speculated  that  animated  presentations 
provide  clear  and  precise  external  illustrations  to  help  students  visualize  physical  laws 
which  involve  changes  in  speed  and  the  path  of  travel.  However,  they  found  that  the 
optimal  use  of  animated  graphics  was  related  to  cognitive  practice  activities  employed  by 
learners.  In  short,  learners1  metacognitive  processes  play  a  much  more  important  role  in 
interpreting  and  reconstructing  their  understanding  of  concepts  than  graphics  per  se.  More 
research  investigating  how  learners  interact  with  graphics  metacognitively  and  how  graphics 
should  be  presented  to  help  the  conceptual  reconstructing  process  is  essential. 

In  some  quarters  visual  thinking  is  considered  an  important  way  of  learning  problem- 
solving  and  promoting  creativity  (O'Connell,  1992).  Reed  (1985)  as  well  as  Treadgill- 
Sowder,  Sowder,  Mover,  and  Moyer  (1985)  have  shown  that  graphic  presentation  of 
mathematical  story  problems  can  lead  to  improved  performance.  However,  creativity  and 
visual  thinking  are  difficult  to  measure.  Although  some  studies  indicate  that  higher  level 
learning  tasks  benefit  most  from  the  graphics  enhanced  instruction,  the  effects  of  graphical 
representations  used  in  higher  level  learning  tasks  are  elusive  and  immeasurable  (Brown, 
1992;  Levie,  1987).  Good  reasoning  skills  depend  crucially  on  the  vigorous  exercise  of 
imagination  (Perkins,  1985).  Graphics  may  provide  support  for  visually  processing 
information  to  generate  multi-dimensional  links  for  imaginative  problem  solution.  In  any 
case,  it  should  be  clear  that  justifying  graphics  in  interactive  learning  environments  goes 
beyond  the  measurement  of  their  effects  on  academic  achievement  (Levie,  1987). 

DIRECTIONS  FOR  FUTURE  RESEARCH 

Empirical  research  in  graphics  has  focused  rather  extensively  on  picture-text  differences 
and  physical  variables  (e.g.,  color  vs.  black  and  white)  to  the  exclusion  of  other  relevant 
factors,  such  as  differences  in  instructional  purposes,  differences  in  graphical  presentations, 
and  the  interactions  among  perception,  learner  characteristics,  and  knowledge  level.  The 
research  cited  above  seems  to  suggest  that  the  optimal  use  of  graphics  is  determined  by 
more  than  visual  factors  such  as  color  cueing,  labeling,  realism,  and  animation.  Research 
based  principles  are  needed  that  describe  the  level  of  practice  permitted  with  graphics, 
variations  among  the  cognitive  tasks  required  in  various  subject  areas,  and  how  learners 
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actually  employ  their  own  strategies  to  mentally  integrate  the  graphics  and  the  images  they 
gen  orate  when  processing  information. 

Of  course,  research  findings  about  whether  graphical  information  is  perceived  and 
remembered  better  than  text,  or  which  format  of  graphics  is  more  efficient  than  others,  are 
always  conditional.  The  condition  is  determined  by  variables  specified  by  the  group  of 
learners  tested,  the  types  of  tasks  given,  the  kinds  of  strategies  instructed,  and  the 
graphical  treatments  in  the  study.  For  example,  although  some  studies  have  found  simple 
line  drawings  more  effective  than  other  formats  for  promoting  learning,  considerable 
variation  exists  among  researchers  in  defining  graphic  formats  from  simple  to  complex, 
(low  realistic  must  a  picture  be  to  be  defined  as  a  realistic  representation  or  how  simple 
must  a  picture  be  to  be  defined  as  a  simple  representation?  More  importantly,  how  can 
these  research  findings  be  applied  in  commonplace  instructional  settings?  The  same  concern 
is  observed  with  respect  to  the  cognitive  practice  given  to  the  learners.  Studies  have  found 
that  embedded  questions  promote  the  use  of  graphics  in  learning  because  they  allow  deeper 
levels  of  cognitive  processing.  However,  how  much  practice  is  enough  for  different  types  of 
learners?  Other  questions  persist.  How  should  questions  be  presented  with  graphics  to 
achieve  a  given  learning  task?  What  level  of  processing  of  graphics  is  actually  employed  by 
learners  in  a  given  learning  task? 

Our  review  of  several  years  of  research  in  graphics  indicates  that  the  effectiveness  of 
visuals  is  not  equally  observed  among  different  groups  of  learners  for  different  kinds  of 
objectives  when  integrated  with  different  kinds  of  instructional  strategies.  The  bottom  line 
appears  to  be  that  even  the  research-based  principles  revealed  by  researchers  in  the  past 
are  specific  to  the  testing  situations  in  which  they  were  found,  with  only  limited  potential  of 
generalizability  to  different  learning  situations. 

Graphics  researchers  try  to  study  the  complexity  of  the  real  world  by  testing  hypotheses 
devised  from  a  narrow  range  of  literature  (Levie,  1987).  "An  aerial  view  of  the  picture 
research  literature  would  look  like  a  group  of  small  tropical  islands  with  only  a  few 
connecting  bridges  in  between."  (Levie,  1987,  pp.  26).  People  who  conduct  picture  research 
usually  ally  their  research  to  some  field,  and  tend  to  focus  on  the  literature  in  their  own 
area,  such  as  perception,  picture  recognition,  picture  memory,  concept  learning,  or  problem- 
solving.  Much  graphics  research  tries  to  answer  questions  from  a  single  perspective. 
Dealing  with  the  complexity  in  learning  in  interactive  learning  environments  is  a  major 
challenge. 

One  of  the  problems  with  the  existing  research  base  is  that  it  has  been  conducted 
according  to  what  Salomon  (1991)  calls  an  analytical  research  approach,  usually  involving 
controlled  experiments.  These  experimental  studies  have  focused  on  assessing  the  effects  of 
discrete  graphical  variables  (e.g.,  color  versus  black  and  white  or  photographs  versus  line 
drawings)  on  a  limited  range  of  learning  outcomes  (e.g.,  memorization). 

Salomon  (1991)  calls  for  increased  systemic  research  in  instructional  technology  and 
related  fields  to  investigate  the  effects  of  instructional  programs  and  products  in  realistic 
settings.  Salomon  says,  "Without  observations  of  the  whole  system  of  interrelated  events, 
hypotheses  to  be  tested  could  easily  pertain  to  the  educationally  least  significant  and 
pertinent  aspects"  (p.  17).  We  argue  that  such  has  been  the  case  with  much  of  the  research 
on  graphics  and  computer-based  learning  environments. 

Of  course,  years  of  experimental  studies  have  yielded  some  generalizable  principles.  For 
obtaining/guiding  attention,  graphics  are  more  easily  perceived  and  more  distinctive  than 
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words  (Paivio,  1990).  (Jrnphics  can  extend  tlu1  time1  readers  arc  willing  to  spend  on  learning 
due  to  the  capability  of  graphics  to  arouse  certain  emotions  (Peeck,  1987).  Based  upon  their 
familiarity,  pictures  have  a  "dual  reality"  for  presenting  objects  themselves  and  for 
functioning  as  surrogates  for  other  abstract  concepts  (Levie,  1987).  Graphics  can  be  direct 
representations  or  analogical  presentations  for  relating  the  unfamiliar  to  the  familiar 
(Alesandrini,  1985).  Graphics  can  support  understanding  of  the  spatial  features  of  an  object 
or  facilitate  the  learning  of  a  sequence  of  procedures  that  might  require  an  awkward  or 
lengthy  description  solely  through  verbal  communications  (Hegarty,  Carpenter,  &  Just, 
1991).  Graphics  can  allow  a  deeper  processing  of  information,  and  thereby  they  can 
reinforce  the  process  of  encoding  (Reid,  Hrigg  &  Beveridge,  1983).  Dwyer  (1970,  1987) 
concluded  that  different  formats  of  graphics  assist  learning  and  comprehension  differently, 
While  these  findings  arc1  encouraging,  they  provide  an  inadequate  foundation  for  guiding 
design  decisions  in  practical  contexts,  especially  with  respect  to  today's  graphical  intensive 
learning  environments,  We  believe  that  future  research  should  be  directed  to  understanding 
"how"  learners  interact  with  graphics  in  using  interactive  learning  environments  that  are 
pertinent  to  their  personal  and  academic  goals. 

THK  NNW  IMPORTANCE  OF  GRAPHICS  RESEARCH 

New  technologies  abound  (e.g.,  digital  movies)  and  thus  many  new  design  elements  are 
being  integrated  into  the  production  of  computer-based  instruction.  The  uses  of  animation 
and  video  capture  techniques,  such  as  (QuickTime,  support  learning  by  allowing  students  to 
view  the!  dynamic  nature  of  processes,  a  factor  limited  in  even  the  best  static  graphics. 
Incorporating  vanmiH  types  of  animal ed  graphic  s  into  interactive  learning  environments  is 
expected  to  support  motivation  and  learning. 

According  to  hiivio's  dual  coiling  theory  (Paivio,  1990),  information  is  easier  to  retain 
when  coded  verbally  and  visually   Within  his  definition  of  dual  coding,  visual  input  also 
refers  to  the  dynamic  features  of  objects  and  events,  Integration  of  animation  features  and 
visual  effects  can  be  an  effective  way  of  arousing  and  maintaining  the  learner's  attention 
during  computer-based  learning  due  to  the  distinctive  cues  provided  in  contrast  to  static 
forms  of  information  and  cueing  stimuli  (Uieher,  19H9a;  Hannifin  &  Peck,  1988). 

Recent  studies  (e.g.,  Brown,  UH)2;  O'Connell,  1992)  have  addressed  the  merits  of 
sophisticated  computer  graphics  in  enhancing  interaction  and  exploration.  Various  graphics 
treatments,  such  as  three-dimensional  modeling,  color,  and  animation,  allow  viewing  from 
different  angles,  so  much  so  that  students  may  perceive  that  they  are  in  virtual 
environments.  Without  in-depth  assessments  using  both  qualitative  and  quantitative 
methods,  research  might  not  come  up  with  guidelines  for  design  of  these  innovative  "high 
tech"  learning  environments. 

Learners  perceive  graphical  information  and  engage  cognitive  processes  which  requires 
them  to  select  relevant  elements  from  the  stimuli  to  attend.  The  use  of  animation,  three-D 
images,  and/or  digital  video  can  help  students  extract  the  central  concepts  within  a  context, 
and  at  the  same  time  conceptualize  what  might  not  be  explained  clearly  in  the  text. 
However,  issues  about  how  graphics  relate  to  the  central  context  to  be  presented,  how  they 
guide  learners  to  comprehend  new  concepts,  and  how  they  can  reinforce  the  process  of 
learning  require  more  research  efforts  aimed  at  providing  a  better  basis  for  incorporating 
state-of-the-art  graphics  into  the  design  of  computer-based  materials. 

In  a  review  of  instructional  visuals,  Dwyer  (19H8)  concludes  that  visualization  provides 
potential  impact  on  increasing  learning  interest,  guiding  thinking,  and  supporting  learning 
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through  reinforcement  and  discrimination  of  relevant  cues.  To  differentiate  the  purposes  of 
using  graphics  in  CHI,  tour  instructional  hi  net  ions  me  used  below,  unhung  attention,  relating 
areas  oHu  mil  unity,  ^uuIinK  comprehension,  and  reinforcing  what  neetlH  to  he  learned,  If 
our  main  research  goal  is  to  help  developers  incorporate  graphics  into  interactive  learning 
environments,  it  may  be  worthwhile  to  look  at  those  functions  individually  in  order  to  help 
focus  research  directions. 

Research  on  Qbtaining/Guidi^  Attention  Through  Graphics  1»  Needed 

Attention-paining  is  an  important  initial  event  of  instruction  (dugne,  Hriggs,  &  Wager, 
1992).  Graphics  usually  convey  non-verbal  interactions  through  their  own  symbolic 
representations.  Due  to  the  differences  between  pictures  and  words  in  the  sensory  and 
physical  features,  pictures  are  more  discriminable  and  more  distinctive  than  words 
(Kobayashi,  1986;  Paivio,  1990).  In  addition  to  the  distinctive  physical  features  of  graphics 
versus  words,  graphics  are  also  often  used  for  arousing  certain  emotional  impact  or  attitudes 
toward  new  content.  Many  pictorial  studies  (e.g.  Levie  &  Lentz,  1982;  Rigney  &  Lutz,  1976; 
Swell  &  Moore,  1980)  support  the  emotional  impact  graphics  have  in  reading.  For  example, 
Poeck  (1987)  argues  that  the  affective  effect  of  pictures  is  due  to  their  features  in 
maintaining  a  high  level  of  concentration  on  symbolic  response  patterns.  Pictures  may  often 
facilitate  in  extending  the  time  readers  are  willing  to  spend  on  learning  (Peeck,  1987).  The 
emotional  effects  of  images  have  been  addressed  in  some  studies,  but  little  research  has 
been  done  in  this  area  with  interactive  learning  environments. 

Much  research  has  been  conducted  to  examine  the  factors  that  influence  the  processes  by 
which  pictures  are  noticed  and  attended  (e.g.,  Beck,  1984;  Butler  &  McKelvie,  1985; 
Jennings  &  Dwyer,  1985).  Physical  characteristics,  such  as  color,  position,  contrast,  motion, 
and  the  size  of  visuals  to  affect  different  communication  levels  arc  emphasized  in  these 
studies  (Goldsmith,  1987).  More  research  needs  to  be  done  to  understand  the  reciprocal 
relations  between  graphics  and  perceptions  of  various  learners  in  order  to  provide 
instructional  designers  with  design  guidelines  for  helping  students  to  see  what  needs  to  be 
seen.  Further,  field-based  research  is  required  to  study  these  variables  within  actual 
instructional  contexts  (in  addition  to  laboratories)  to  determine  how  learners  attend  to 
information  through  the  use  of  graphics. 

Research  on  Relating  Areas  of  Familiarity  Through  Graphics  Is  Needed 

Graphic  representations  can  convey  information  rapidly  and  can  be  recalled  rapidly  due 
to  their  "dual  reality,"  i.e.,  they  represent  objects  themselves,  and  they  function  as 
surrogates  for  other  concrete  objects  or  even  abstract  concept  (Levie,  1987).  When  a  picture 
is  used,  it  is  perceived  and  interpreted  by  viewers  based  on  their  own  understanding  and 
experience  in  constructing  a  meaningful  representation.  Pictures  may  be  direct  portrayals  or 
analogical  representations  used  to  relate  what  learners  already  know  to  the  understanding 
of  new  concepts.  Although  some  fundamental  skills  for  visual  recognition  are  innate, 
decoding  pictorial  information  is  related  with  picture-viewing  experience,  culture-bound 
conventions,  and  knowledge  experience  (Levie,  1987). 

Since  we  use  images  to  communicate,  we  need  to  know  how  to  t  Voose  the  most  valuable 
images  to  express  ideas  (Nanny,  1990).  Jn  interactive  learning  env   mments,  one  of  the 
mainstays  of  graphical  user  interfaces  is  to  use  the  appropriate  gra>  nical  representation  to 
present  a  resource,  an  option  or  an  action  (Cates,  1993).  Using  appropriate  pictures  for 
different  audiences  at  the  right  time  is  challenging.  More  research  needs  to  be  done  to 
observe  how  people  from  different  fields,  different  levels  of  expertise  in  certain  areas,  or 
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different  age  groups  perceive,  interpret,  and  interact  with  the  graphics  in  a  learning 
environment. 

Research  on  Guiding  Comprehension  Through  Graphics  Is  Needed 

Using  graphics  for  guiding  comprehension  is  widely  used  among  scientific  and  technical 
information.  For  examples,  Rieber  and  his  colleagues  used  animuted  graphics  in  presenting 
an  introduction  to  Newton's  Law  of  Motion  (Rieber,  1989b;  Rieber,  1990a,  b;  Richer,  Boyco, 
&  Assad,  1990;  Rieber  &  Hannafin,  1988).  During  recent  years,  there  has  been  a  growing 
interest  and  development  effort  for  graphical  innovations.  These?  technical  advances  make  ' 
the  communication  of  the  information  available  in  a  wide  variety  of  approaches,  The  visual 
properties  of  graphics  help  learners  understand  spatial  features  of  an  object  or  a  sequence  of 
procedures  that  might  require  an  awkward  or  lengthy  description  solely  through  verbal 
communication  (Hegarty,  Carpenter,  &  Just,  1991),  However,  it  is  also  easy  "to  be 
superficial  to  dazzle  someone  with  beautiful  graphics"  when  using  technology  (l)aro,  1992). 
Research  or.  bow  graphics  can  be  presented  to  make  the  abstract  and  complex  concepts 
concrete  ami  simplified  is  needed. 

In  addition  to  conveying  information  that  is  spatially  related,  the  use  of  graphics  also 
provides  various  types  of  assistance  in  organizing  concepts  and  principles  into  elaborate 
networks  (e.g.,  mental  models).  Mental  models  are  used  to  integrate  information  into  a 
holistic  perspective  or  useful  strategy.  The  use  of  graphics  as  an  organizer  has  been  shown 
to  be  an  effective  thinking  tool  and  widely  applied  in  various  subjects  for  both  inductive  and 
deductive  thinking  (e.g.,  Clarke,  1991;  Tajika,  Taniguchi,  Yamamoto,  &  Mayer,  1988; 
Krahn  &  Blanchaer,  1986).  Graphics  used  for  this  purpose  provide  learners  ways  to 
visualize  and  control  their  thinking  about  content  in  the  subject  areas.  However,  studies 
designed  to  observe  and  contrast  the  visualization  processes  used  by  experts  and  novices  as 
they  construct  mental  models  are  needed  (Friedman,  1993,  Jih  &  Reeves,  1992). 

Most  research  is  conducted  to  compare  and  analyze  the  outcomes  among  specific 
graphical  treatments  and  other  types  of  treatments.  Little  research  has  been  done  to  build 
understanding  of  the  strategies  students  actually  employ  to  construct  understanding  from 
graphics.  More  research  needs  to  be  done  through  documenting  the  processes  employed  by 
students  to  learn  new  knowledge,  skills,  and  attitudes  through  the  use  of  graphics. 

Research  on  Reinforcing  Learning  Through  Graphics  Is  Needed 

Pictorial  stimuli  are  used  to  help  readers  emphasize  concepts  to  be  learned  (Beck,  1987). 
Pictures  are  often  used  in  facilitating  the  encoding  process  (Levie  &  Lentz,  1982;  Reid,  Brigg 
&  Beveridge,  1983).  It  is  considered  that  when  presented  together  with  text,  more  cues  are 
provided  to  allow  deeper  processing  of  the  intended  information.  The  provision  of  graphics 
also  permits  learners  to  interact  with  the  materials  with  different  dimens:  <ns,  which 
encourages  the  information  to  be  processed  in  a  more  elaborated  way.  Used  as  an  encoding 
cue,  graphic  representations  provide  interpretation  to  given  verbal  information  so  that  the 
information  can  be  communicated  more  explicitly  and  accurately  (Morris  &  Hampton,  198U; 
Alesandrini  &  Rigney,  1981).  With  denotation  of  concrete  representations  to  textual 
materials,  the  information  is  more  easily  retained  in  memory  because  both  verbal  and  image 
systems  are  activated  in  storing  the  information.  Under  the  assumptions  of  dual  coding 
(Paivio,  1990),  when  information  is  presented  verbally  and  visually,  it  can  be  retained  longer 
in  memory  because  both  memory  systems  are  used. 

To  reinforce  learning,  students  need  to  deeply  process  the  information  through  the  use  of 
their  own  cognitive  strategies  to  relate  both  verbal  and  visual  information  from  the 
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instruction.  Research  shoald  focus  on  how  information  should  be  presented  and  wlttit  other 
instructional  strategies  should  be  included  to  promote  the  use  of  students'  own 
motacognition  for  deeper  processing. 

NEW  DIRECTIONS  FOR  GRAPHICS  RESEARCH 

Six  recommendations  are  made  to  guide  graphics  research  in  interactive1  learning 
environments: 

1 .  A  variety  of  research  paradigms  should  be  explored  in  conduct  i.ig  graphics  research 
(e.g.,  a.  positivist,  quantitative,  b.  interpretivist,  qualitative,  and  c.  criltciil  tlienrv, 
postmodernist).  Studies  employing  multiple  methods  should  he  considered, 

2.  Researchers  should  attempt  to  ground  their  research  in  broad,  ceaiprelieiiHive 
theories,  of  human  perception,  memory,  and  learning.  Further,  research  that  does  not  reflect 
a  deep  appreciation  of  the  theory  and  research  on  graphics  that  has  huilt  up  aver  the  past 
fifty  years  will  likely  rediscover  what  is  already  known  or  lead  to  irrelevant  cone ItiHioiiH, 

3.  Subjects  in  graphics  studies  should  be  involved  in  learning  that  is  personally 
meaningful  and  has  real  consequences  for  them. 

4.  Graphics  research  should  employ  commercially  viable  interactive  products  or  products 
that  are  under  development  as  opposed  to  the  "nonsense"  materials  used  in  some  earlier 
studies.  The  continued  use  of  irrelevant  or  content-free  materials  -s  highly  unlikdy  to 
provide  us  with  useful  principles  for  design  of  interactive  learning  onv>  lOnmonts. 

5.  Although  laboratory  studies  can  be  useful,  field-b^Fed  research  should  be  preferred  to 
increase  the  generalinability  of  research  findings. 

6.  Regar'Vi^ss  ox  waeiher  research  is  conducted  in  labs  or  the  field  (e.g.,  schools,  training 
centers,  businesses,  tb 1  research  protocols  used  by  soft'vuie  usability  testing  laboratories 
may  provide  excellent  [  rldarce  for  the  conduct  of  these  s*  lies  (Nielsen,  1993).  (The 
Learning  and  Performance  Support  Laboratory  at  The  University  of  Georgia  is  establishing 
two  facilities,  one  fixed  and  one  mobile,  to  employ  usability  testing  strategies  in  research  on 
graphics,  navigation,  and  mental  models  in  interacts  >  learning  environments  and 
performance  support  systems.) 

SUMMARY 

Substantial  research  on  graphics  using  a  wide  variety  of  media  has  provided  a 
foundation  for  further  research  that  may  provide  an  enhanced  bas^s  for  guiding  the  design  of 
interactive  learning  environments.  However,  we  maintain  that  future  research  demands 
new  directions  in  terms  of  both  the  questions  asked  and  the  methods  used  by  graphics 
researchers.  Questions  should  be  focused  en  understanding  "how"  learners  process  graphics. 
Methods  nhould  ho  expanded  to  include  a  wider  spectrum  of  research  paradigms.  All 
graph ich  remvM'ch  should  be  guided  by  ir-  depth  understanding  of  theory  and  research 
related  to  pureej rtion,  memory,  learning,  and  other  cognitive  processes. 
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Introduction 


Compressed  video  is  not  a  new  technology.  As  early  as  1970  AT&T  produced  its 
Picturephone  which  was  based  on  its  experiments  with  reduced  bandwidth  and  the  use  of 
digitized  video.  However  its  cost  and  complexity  thwarted  growth  and  demand  for  the 
product  until  the  mid  to  late  1980s,  and  now,  "compressed  video  is  a  hot  technology  and 
will  get  hotter  throughout  the  1990s.  Increased  microprocessor  speeds,  decreasing  costs  and 
the  over  increasing  availability  of  bandwidth  are  all  driving  the  usefulness  and  economics  of 
the  technology.  The  underlying  technology  of  compressed  video  is  complex,  but  so  is  computer 
and  telephone  technology.  At  the  practical  level,  implementation  and  operation  of 
compressed  video  is  not  more  complicated  than  muny  computer  networks  ,  and  u  lot  less 
complex  than  some."  (Burton,  1993)-   This  paper  presents  the  results  of  three  surveys  about 
live,  two-way  interactive  video  (CV)  and  discusses  some  possible  trends  in  utilization, 
applications  and  the  technology  itself.  The  surveys  are  drawn  from  an  AECT  survey  done  by 
the  authors,  the  International  Video  Teleconferencing  Source  Hook  (hereinafter  referred  to  as 
The  Source  Book  ),  and  the  National  Association  of  State  Telecommunications  Directors 
(NASTD)  1993  survey  of  the  western  region  of  the  United  States. 

In  February  of  1993,  AKCT  published  Compressed  Video;  Operations  and 
Applications  by  Barbara  Hakes,  Steven  Sachs,  Cecelia  Box  and  John  Cochenour.  The  book 
contains  user  profiles  from  some  twenty  different  institutions,  businesses  and  agencies. 
During  the  time  the  book  was  being  published,  and  since  its  publication,  the  use  of  live, 
two-way  compressed  video  (CV)  has  been  increasing  dramatically.  As  a  result,  a 
commitment  to  produce  a  supplement  to  the  book  was  made  with  a  target  publication  for 
1994.  In  order  to  develop  the  supplement  it  was  necessary  for  the  authors  to  conduct  a 
survey  of  compressed  video  users  from  higher  education,  K-12  and  the  private  sector.  This 
paper  presents  the  initial  results  of  that  survey  which  is  still  in  progress. 

The  first  edition  of  the  Source  Book  was  published  by  AT&T  Global  Video  Services 
and  the  International  Teleconferencing  Association  in  1992;  and  the  second  edition  in  1993. 
The  Source  Book  was  the  first  early  attempt  to  compile  a  data  base  of  compressed  video 
information.  The  1992  edition  lists  1,600  entries  world  wide  for  video  conferencing  facilities. 
The  1993  edition  lists  2,381  entries.  Additionally,  it  provides  the  following  information:  1) 
whether  the  facility  is  a  private  or  public  room,  2)  the  model  of  codec  being  used,  3) 
transmission  rates,  4)  encryption  type,  5)  carrier  provider,  6)  video  phone  number,  if 
appropriate,  and  7)  a  contact  person  and  the  contact  person's  voice  phone  number. 

The  NASTD  survey  was  conducted  by  Dennis  Nincehelser,  Director  of  the  Bureau  of 
Administration  in  the  division  of  Information  Services  in  the  state  of  South  Dakota.  The 
survey  examined  the  compressed  video  operations  and/or  plans  of  the  16  states  in  the 
Western  Interstate  Commission  for  Higher  Education  cooperative:  North  Dakota,  South 
Dakota,  Nebraska,  Montana,  Wyoming,  Colorado,  New  Mexico,  Idaho,  Utah,  Arizona, 
Nevada,  *Vashington,  Oregon,  California,  Alaska  and  Hawaii.  The  results  of  that  survey, 
are  discussed  in  this  paper  as  well. 

Procedure 

The  findings  of  the  aforementioned  sources  are  reviewed  and  apparent  trends  in 
compressed  video  utilization,  applications  and  the  technology  itself  are  noted.  For  the  AKCT 
survey,  the  authors  used  the  1992  edition  of  the  Source  Book  published  by  AT&T  Global 
Video  Services  and  tho  Internationa]  Teleconferencing  Association  as  the  population  base. 
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In  June  1993,  surveys  were  mailed  to  the  938  entries  listed  in  The  Source  Book.  In  July  of 
1993,  follow  up  letters  were  mailed.  This  letter  requested  a  reply  by  September  15,  1993. 
In  November  another  survey  was  mailed  to  those  not  responding  by  the  September  deadline. 
During  December,  various  smaller  mailings  were  made  to  additional  institutions  as  the 
authors  became  aware  of  their  uses  of  CV  through  a  variety  of  other  sources.  Surveys  were 
also  completed  via  telephone  interview  during  December  1993.  A  total  of  44  surveys  were  , 
completed  via  these  processes 

Multiple  sources  to  examine  trends  for  this  paper  were  used  for  the  following 
reasons.  The  AECT  survey  is  not  yet  complete  and  response  rate  has  been  low.  Data  for 
The  Source  Book  ,  although  voluminous,  was  collected  in  1990  to  1991  for  the  1992  edition 
and  1992  to  1993  for  the  latest  edition.  Compressed  video  technology  has  changed 
dramatically  since  the  time  of  the  first  edition  and  new  users  and  applications  are  emerging 
as  this  paper  is  being  written.  In  fact,  "the  anticipated  growth  of  video  teleconferencing  in 
the  1990s  is  similar  to  the  FAX  'explosion'  of  the  1980s"  (Burton,  1993).  Also  information 
on  data  collection  for  The  Source  Book  is  sketchy,  and  the  authors  admit  that  it  may 
represent  a  biased  picture  in  unknown  ways.  For  example,  while  the  1993  edition  lists  the 
state  of  Wyoming  as  having  two  video  conferencing  facilities  equipped  with  CLI  it  in  fact 
supports  twelve  video  conference  facilities  equipped  with  VTel  codecs.  The  NASTD  survey  is 
restricted  to  the  Western  region  and  may  also  reflect  some  bias  because  of  its  geographic 
restrictions.  Because  of  possible  inaccuracies  in  each  survey,  wt  have  triangulated  the 
surveys  reported  here  in  order  to  identify  possible  trends. 

Findings 

AECT  Survey 

While  the  authors  are  not  yet  finished  with  data  collection,  the  information  that  has 
been  obtained  reveals  some  interesting  trends  and  possible  patterns  for  the  future  direction 
of  compressed  video  conferencing  and  networking.  Among  22  questions  responded  to  in  the 
survey  were  the  following.  * 
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When  did  vuu  start  usuiy  cuinorusMed  video?  The  earliest  start  date  was  1980. 
Most  of  the  respondents  begun  in  either  1 1)1)0  c»'  I  Mil.  Thin  probably  reflects  the  use  of  the 
first  edition  Sourcv  Hook  ns  the  population  hum1  for  the  basic  survey.  The  first  edition  used 
data  compiled  in  191MMM  and  the  institutions  listed  therein  would  have  begun  prior  to  its 
publication.  The  institutions  starting  in  1992-iM  as  shown  in  our  data  come  from  a  smaller 
database  compiled  by  the  authors  own  networking,  (sue  figure  1) 

How  did  you  decide  to  use,  compressed  video  us  opposed  to  other  telecommunications 
systems?  Eighteen  percent  of  the  respondents  did  not  provide  an  answer  to  this  question, 
but  those  that  did  had  an  interesting  variety  of  reasons  for  choosing  compressed  video.  Fifty 
percent  of  the  group  listed  cost  as  the  primary  reason  for  selecting  compressed  video.  The 
other  two  major  categories  were  the  higher  quality  of  the  compressed  video  (23%)  and  the 
existence  of  a  system  (18%)  where  the  system  was  inherited  and  original  reasons  were 
unknown,  (see  figure  2)  It  should  be  noted  that  in  some  cases  compressed  video  and  other 
telecommunications  systems  are  used  in  conjunction,  and  it  is  not  necessarily  a  decision  to 
use  only  one  system. 


Cost     Avail.  Existing     CV       User   product-Environ-  Ease  of  No 

location  system  quality  request    ivity    mental    Tech.  response 

Figure  2:  Reasons  for  choosing  compressed  video  over  other  telecommunication  systems 


What  delivery  svstem(s)  do  you  use  (i.e.  satellite,  land  line  (fiber.  Tl  etc.)?  Many  of 
the  systems  described  by  our  respondents  used  more  than  one  delivery  system,  so  the 
percentages  listed  here  exceed  100  percent.  Eighty  four  pereent  of  the  respondents  stated 
they  were  using  Tl  as  a  delivery  system.  Twenty  three  percent  wore  UHing  satellite  delivery; 
eleven  percent  were  on  land  line,  and  five  percent  were  using  switched  56.  Thir  five  percent 
are  primarily  represented  by  institutions  that  begun  using  com preswd  video  during  199:!. 

How  many  sites  are  included  in  the  network?  Two  of  the  responses  listed  only  one 
site,  or  essentially  not  part  of  a  yet  established  network.  One  response  listed  1400  sites  in 
the  network.  However,  the  majority  of  the  respondents  listed  sites  that  numbered  between 
six  and  twenty,  (we  figure  8) 
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Figure  3:  Number  of  compressed  video  sites  in  the  network 


Please  describe  each  application  that  is  presently  using  the  compressed  video 
netwqrk.  Several  different  applications  were  listed.  Most  of  the  respondents  had  more  than 
one  application.  The  applications  have  been  categorized  and  the  percentage  of  respondents 
using  the  application  are  shown  in  figure  4. 

Who  operates  the  facilities?  Again,  this  was  not  an  either/or  question  and  many  of 
our  respondents  had  more  than  one  means  of  operating  the  system.  Sixty  eight  percent 
stated  they  used  a  technician  at  least  part  of  the  time  to  handle  the  operation  of  the  system. 
Fifty  seven  percent  stated  the  primary  user  or  a  participant  operated  the  system  at  least 
part  of  the  time.  Seventy  percent  of  the  respondents  indicated  that  the  system  operator  had 
received  some  training  regarding  the  operation  of  the  system.  The  most  common  training 
was  simply  an  audio/visual  orientation,  hut  two  percent  of  the  operators  had  electronics 
training  and  nine  percent  had  video  production  training. 

What  rate(s)  do  vou  use  for  transmission?  The  tnoHt  common  transmission  rate 
reported  was  384  kbps.  Several  of  the  sitea  reported  more  than  one  rate  was  available  for 
use  depending  upon  the  connecting  network  and  the  application  need,  (see  figure  5) 
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Figure  4:  Applications  for  compressed  viilou  miii  t In*  |><kr<i<ntfiK«  of 
respondents  using  such  iippliniliiiim 

Are  vou  presently  using  (or  plan  to  use)  the  public  a  witched  fcti  network  for  interstate 
or  international  communications?  Thirteen  of  the  respondents  fin  led  to  answer  this  question 
and  fifty  seven  percent  responded  negatively.  However,  thirty  percent  of  the  survey 
respondents  indicated  they  were,  or  had  plans  to,  use  puhlie  switched  Bfl  network  for 
interstate  or  international  communications. 

What  hardware  model  and  brand  are  you  using?  Our  findings  were  similar  to  those 
in  The  Source  Book,  (see  discussion  below)  The  more  popular  codec  brands  were: 
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VideoTelecom  •  48%,  CM  •  ,ri4%,  and  PictureTel  I IV  Stxf i-i-n  percent  oflhe  respondents 
wore  scattered  anions  other  systems.  The  respondents  were  fairly  evenly  divided  between 
stationary  and  rollabout  systems. 
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112        128       256        384       768       786       1 344  No  response 

Transmission  Rate  -  kbps 


Figure  5:  Percentage  of  respondents  reporting  use  of  a  specific  transmission  rate. 


What  peripherals  are  attached  to  the  system?  Two  popular  peripherals  were  a  VCR 
and  the  Elmo  presenter,  both  were  part  of  fifty  seven  percent  of  the  systems.  The  most 
common  peripheral,  however,  was  a  computer.  Sixty  four  percent  of  the  systems  had  a 
computer  as  part  of  the  system,  and  the  MS-DOS  compatible  computer  was  used  seventy 
five  percent  of  the  time.  Additional  peripherals  that  were  frequently  listed  included  external 
cameras,  fax  machines,  and  the  pen  pal.  Other  peripherals  that  were  less  commonly 
mentioned  included:  slide  projectors,  overhead,  CD-Rom,  speakers,  zoom  box,  black  box 
converter,  special  effects  switcher,  and  videowriter. 

Source  Book 

Educational  institutions-  1992/3.  The  1993/11  Stmrvv  lUmk  contains  1.600  listings 
for  international  facilities  for  compressed  video.  Facilities  in  the  United  States  comprised 
about  938  listings  for  approximately  M)%  nf  the  sitim  available  wtirld  wide.  121  sites  were 
not  included  in  the  authors*  analysis  due  to  incomplete  infill  unit  inn  contained  in  the 
directory.  Of  the  933  sites  listed  in  the  United  states  iVIH  were  listed  as  private  facilities 
and  only  260  locations  were  listed  as  public,  (see  f  igure  fi)  Kdlinitiminl  institutions 
supported  only  28  of  the  facilities  listed  in  the  directory  far  a  little  less  than  30%  of  the 
total  number  of  compressed  video  locations.  The  twenty  eight  educational  institutions  listed 
in  the  source  book  utilized  22  sites  with  Compression  Labs  lue.«'Ll)  codecs,  four  with 
PictureTel  (PCTL)  Codecs  and  six  with  VideoTelecom  (VTKL)  codecs,  (see  figure  7)  One  site 
utilized  a  GPT  codec  and  the  28th  site  utilized  a  VisuaLilik  codec.  Six  sites  supported  two 
different  vendors'  codecs  operating  at  different  transmission  speeds.  Transmission  rates 
(see  figure  8)  at  these  sites  ranged  from  nine  locations  which  used  a  full  Tl  earrior  (1.544 
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megabytes  p  r  second),  three  locations  which  used  variable  rates  from  384  Kbps  to  1.544 
Mbps,  one  site  using  1 12  Kbps  to  1.544  Mbps,  two  which  used  768  Kbps,  eight  which  used 
384  Kbps,  five  which  used  112  Kbps.  Since  lower  transmission  rates  require  less  space  on 
the  carrier,  whether  it  be  land  line,  satellite  or  microwave,  conferencing  costs  are  reduced  at 
the  lower  rates.  Yet  at  the  lower  rates  some  quality  in  transmission  is  sacrificed. 
Transmission  at  384  Kbps  utilizes  only  about  1/3  of  the  capacity  of  a  Tl  carrier.  In  the 
sample  of  educational  institutions  contained  in  The  Source  Book,  40%  utilized  transmission 
rates  at  768  Kbps  or  a  full  Tl,  14%  at  variable  rates  ranging  from  1 12  Kbps  to  a  full  Tl , 
and  46  %  used  transmission  rates  at  384  Kbps  or  lower. 


Private   Public    Private    Public   Private   Public    Private  Public 
1992  1993  1992  1993 

Education  Sites  Non-education  Sites 


Figure  6:  Percentage  of  private  and  public  facilities  for  educational  and  non- 
educational  compressed  video  sites,  1992  and  1993. 


Educational  Institutions- 1993/4.  The  1993/4  Source  Book  contains  2,381  listings 
for  international  facilities  for  compressed  video.  This  represents  an  increase  in  listing  of 
38%  over  the  previous  edition.  Facilities  in  the  United  States  comprised  1,526  listings  for 
approximately  64%  of  the  sites  available  world  wide.  64  sites  were  not  included  in  the 
authors'  analysis  due  to  incomplete  information  contained  in  the  directory.  Of  the  1,562 
sites  listed  in  the  United  states  1,004  were  listed  as  private  facilities  and  521  locations 
wore  listed  as  public,  (see  figure  6)  This  increase  in  the  number  of  public  compressed  video 
facilities  is  double  the  number  listed  in  the  previous  year's  directory.  Educational 
institutions  supported  49  of  the  facilities  listed  in  the  directory  for  a  little  less  than  32%  of 
the  total  number  of  compressed  video  locations.  The  forty-  nine  educational  institutions 
listed  in  the  source  book  utilized  eighteen  sites  with  Compression  Labs  Inc(CLI)  codecs, 
eight  with  PictureTel  (PCTL)  Codecs  and  thirty-seven  with  VideoTelecom  (VTEL)  codecs. 
Three  sites  utilized  GPT  codecs,  (see  figure  7)  Seventeen  sites  supported  two  different 
vendors'  codecs  operating  at  different  transmission  speeds.  Transmission  rates  (see  figure 
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8)  at  these  sites  ranged  from  six  locations  which  used  a  full  Tl  carrier  (1,544  megabytes  \wr 
second),  two  sites  using  112  Kbps  to  1,544  Mbps,  four  which  used  768  Kbps,  twenty-two 
which  used  384  Kbps,  thirteen  of  which  used  112  Kbps,  In  the  sample  of  educational 
institutions  contained  in  The  1993/4  Source  Book,  20%  utilized  transmission  rates  at  7f>H 
Kbps  or  a  full  Tl,  8%  at  variable  rates  ranging  from  112  Kbps  to  a  full  Tl,  and  71  %  used 
transmission  rates  at  384  Kbps  or  lower. 
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Figure  8:  Transmission  rates  for  educational  compressed  video  niton,  I  OIK!  and 

1993. 

U.  S.  Non-education  Compressed  Virioo  Sites- 1992/3.  In  examining  Thv  Source 
Hooks  listings  for  all  938  compressed  video  installations  in  the  United  States  the  following 
observations  were  made.  Some  sites  supported  more  than  one  codec  ciporut inu  at  different 
transmission  rates.  According  to  entries  made  in  The  Source  nook  1121  situs  did  not  provide 
i  n  formation  either  on  the  model  of  codec  or  transmission  rates  being  used.  Of  those 
providing  complete  information,  789  listed  one  or  more  brands  of  codecs.  Thirty  four  sites 
supported  at  least  two  models  of  codecs  and  four  sites  supported  three*  or  more  brands  of 
codecs.  The  latter  were  primarily  communications  industries  groups  supporting  public 
compressed  video  rooms.  Of  the  789  listings  providing  codec  information  H2%  (581) 
supported  CM  codecs;  16%  (145)  supported  PictureTel;  9%  (86)  supported  VideoTelecom; 
7%  (61)  used  GPT,;  and  less  than  1%  varyingly  used  Nec,  Mitsubishi,  Vistacom,  BBT  or 
(1EC  codecs,  (see  figure  7)  Transmission  rates  (see  figure  9)  were  reported  in  836  of  the 
entries.  The  following  general  pattern  emerged  after  analyzing  the  entries.  Approximately 
Forty-five  percent  of  the  CV  sites  transmit  at  384  Kbps  or  lower,  and  fifty-  five  transmit  768 
Kbps  or  higher 
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Figure  9:  Transmission  rates  for  non-educational  compressed  video  sites,  1992 

and  1993. 

U.  S.  Non-oducation  Compressed  Video  Sites- 1993/4.  In  examining  The  Source 
Hooks  for  thti  1993/4  edition  listings  for  all  1,526  compressed  video  installations  in  the 
United  States  the  following  observations  were  made.  Many  sites  supported  more  than  one 
coder  operating  at  different  transmission  rates.  According  to  entries  made  The  Source  Book 
M  HitvH  did  not  provide  information  either  on  the  model  of  codec  or  transmission  rates  being 
used,  but  142H  nites  listed  one  or  more  brands  of  codecs.  Fifty-one  sites  supported  at  least 
two  models  of  codecs  and  eleven  sites  supported  three  or  more  brands  of  codecs.  The  latter, 
ngnin,  were  primarily  communications  industries  groups  supporting  public  compressed  video 
rooms.  Of  the  1,428  complete  listings  providing  codec  information  44%  (644)  supported  CI  J 
codecs;  35%  (524)  supported  PictureTel;  12%  (86)  supported  VideoTelecom;  5%  (74)  used 
OPT;  and  less  than  1%  varyingly  used  Nec,  Mitsubishi,  Vistacom,  BBT  or  GEC  codecs,  (see 
figure  7)  Transmission  rates  (see  figure  9)  were  reported  in  1,428  of  the  entries.  The 
following  general  pattern  emerged  after  ana.,  zing  the  entries.  Approximately  Sixty-six 
percent  of  the  CV  sites  transmit  at  384  Kbps  or  lower,  and  forty-four  percent  transmit  768 
Kbps  or  higher. 

NASTD  1993  Survey  of  the  Western  Region 

During  the  spring  and  summer  of  1993,  Dennis  Nincehelser,  Manager  of  Data,  Video 
and  Telecommunications  in  the  Bureau  of  Administration  in  the  Division  of  Information 
Services  for  the  state  of  South  Dakota  conducted  a  survey  of  compressed  video  operations  or 
plans  of  the  16  'iiaies  in  the  Western  Interstate  Commission  for  Higher  Education 
cooperative.  Dennis  granted  permission  to  include  the  results  of  that  survey  in  this  paper. 

A  summarizing  discussion  of  the  data  is  provided  here.  Of  the  16  western  states 
represented  in  che  survey,  Three  are  planning  for  the  use  of  video  teleconferencing  and  seven 
are  ulrcady  using  it.  Of  the  10  states  responding  to  the  survey,  two  use  broadband  analog 
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video  conferencing.  One  of  these  states  is  in  the  process  of  changing  to  compressed  digital 
video.  Four  states  use  CLI  based  systems  operating  at  variable  transmission  rates  while 
three  states  us  VTEL  codecs  operating  at  384  Kbps  or  lower  and  one  additional  state  is 
planning  to  expand  its  use  of  two  VTEL  codecs  statewide.  Nine  of  the  states  either  use  or 
plan  on  using  video  bridges.  Five  of  the  states  plan  to  use  inverse  multiplexers,  three  are 
considering  the  use  of  this  technology  and  two  are  not  planning  to  use  it.  Higher  education 
represents  the  majority  of  use  in  all  but  three  of  the  states  where  state  government  is  the 
biggest  user.  However,  those  networks  with  a  preponderance  of  use  from  higher  education 
also  share  the  network  with  state  government  and  in  some  cases  with  K-12  and  the  private 
neel  or. 


Educational  Institutions.  Based  on  the  data  contained  in  two  editions  of  The  Source 
Book,  it  would  appear  that  there  is  a  trend  toward  the  use  of  lower  transmission  rates 
among  educational  institutions.  There  were  increases  of  9%  and  22%  respectively  in  the  use 
of  1 12  Kbps  and  384  Kbps  transmission  rates.  There  was  a  decrease  of  16  %  in  the  use  of 
Tl  transmission.  In  the  first  edition  of  The  Source  Book  only  46%  of  educational  institutions 
used  a  transmission  rate  of  384  Kbps  or  lower.  In  the  current  edition,  this  percentage  has 
increased  to  71%. 

Relative  to  the  codecs  being  used  at  educational  institutions  when  comparing  the 
data  presented  in  the  two  editions  ,  there  is  an  increase  of  39%  in  the  use  of  VideoTelecom 
(VTEL)  equipment,  a  decline  of  36%  in  the  use  of  CLI  and  no  change  in  the  percentage  of  use 
of  PictureTel  codecs  while  GPT  codecs  rose  approximately  2%  and  VSI  declined  by  3%. 

Non-Educational  CV  Sites  .  Based  on  the  data  contained  in  two  editions  of  The 
Source  Book,  it  would  appear  that  there  is  a  trend  toward  the  use  of  lower  transmission 
rates  among  non-education  CV  users.  There  were  increases  of  2%  and  22%  respectively  in 
the  use  of  56  Kbps  and  112  Kbps  transmission  rates.  There  were  decreases  5%  in  the  use 
of  384  Kbps,  and  12  %  in  the  use  of  768  Kbps  and  10%  in  Tl  transmission.  In  the  first 
edition  of  The  Source  Book  only  4fi%  of  the  CV  sites  used  a  transmission  rate  of  384  Kbps  or 
lower.  In  the  current  edition,  thiH  percentage  has  increased  to  66%. 

Relative  to  the  codecs  being  used  at  these  sites,  when  comparing  the  data  presented 
in  the  two  editions  ,  there  are  increase  of  19%  in  the  use  of  PictureTel  codecs;  3%  in 
VideoTelecom  (VTEL)  equipment,  and  declines  of  18%  in  the  use  of  CLI;  2%  in  GPT  and  3% 
in  NEC  and  no  change  in  the  percentage  of  use  of  other  vendors1  products. 

While  CLI  codes  represent  the  largest  number  (644)  in  the  latest  directory  fany  of 
these  codecs  were  sold  early  to  large  clients  such  as  SPRINT,  Rockwell  International,  (IK, 
Sears  Technology,  the  U  S  Armed  Services  and  others.  Sales  to  these  clients  alone  totaled 
some  250  units,  all  old  model  Rembrandt  operating  and  768  Kbps  or  full  Tl.  PictureTel 
has  temporarily  gained  a  huge  share  of  the  market  with  its  low  bandwidth  codecs.  Hut.  CLI 
has  countered  with  the  Eclipse  and  VTEL  also  has  announced  low  bandwidth  products.  In 
noting  the  higher  percentage  use  of  VTEL  products  in  educational  institutions  the  authors 
believe  that  the  increased  popularity  of  the  VTEL  line  in  education  may  be  due  to  the 
integrated  multimedia  design  of  the  System.  In  interviews  with  educational  users  they  are 
quick  to  point  out  the  advantages  of  a  multimedia  system  in  educational  environments.  Hut 
the  trend  does  appear  to  be  toward  lower  bandwidth  technology. 


Trends  and  Results 
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Abstract 


This  paper  presents  a  cognitive  model  of  journal  writing  as  a  metacognitive  tool  in  un- 
derstanding stories  in  an  introduction  to  literature  course. 

The  methodology  was  primarily  qualitative.  Data  included  students'  journals  and  class 
comments;  surveys  of  the  helpfulness  of  journal  writing  ;  individual  learner  characteristics 
(e.g.,  reading  ability,  field  dependence/  independence,  academic  record,  course  grade,  and 
confidence  in  journal-writing  ability);  and  case  studies  of  five  students.  Findings  indicated 
that  (a)  students  generally  viewed  journal  writing  as  a  flexible  cognitive  tool  which  helped 
them  construct  the  meaning  of  stories,  and  (b)  writing  journals  scaffolds  (supports)  students 
in  attending  to  details,  asking  questions,  and  answering  their  own  questions.  Although 
high-ability  readers  tended  to  engage  in  different  cognitive  activities  than  lower-ability 
readers,  frequencies  of  questions  and  self-explanations  were  not  related  to  story  difficulty. 
Students1  valuations  of  journal  writing  were  not  significantly  related  to  measures  of 
achievement  (course  grades,  semester  grade-point  averages,  or  cumulative  grade-point 
averages). 

Moreover,  journal  writing  is  a  very  dynamic  process;  no  single  cognitive  model  can 
describe  journal  writing  either  within  or  across  students.  A  two-part  model  was  developed 
to  describe  the  process:  (a)  factors  which  influence  journal  writing  (task  difficulty,  individual 
learner  characteristics,  teacher  expp  ations,  student  strategies,  external  resources,  overt 
activities),  and  (b)  components  of  journal  writing  (establishing  a  goal,  constructing  the 
textbase,  constructing  the  situation  model,  predicting  outcomes,  identifying  significant 
elements,  reflecting  on  meaning,  and  assembling  the  schema). 

Recommendations  for  research  are  included. 

Introduction 

Journal  writing  has  been  used  as  an  instructional/learning  strategy  in  a  wide  variety  of 
educational  settings  from  first-grade  mathematics  (Wnson-EUam,  1987)  to  teacher  education 
(Bean  &  Zulich,  1989).  The  type  of  journals  which  are  the  focus  of  this  paper  are  more  pre- 
cisely called  "academic  journals/1  which  are  "usually  based  on  responses  to  assigned 
readings  or  topics  presented  in  class,  and  are  most  often  evaluated  not  for  their  style  or 
control  of  formal  writing  abilities  but  for  their  reflection  of  students'  learning  and  thinking" 
(Anson  &  Beach,  1990,  p.  3). 

Although  there  is  a  strong  theoretical  base  for  the  use  of  academic  journals  as  an 
instructional/learning  strategy,  there  is  little  research  to  inform  instructional  applications. 
The  literature  on  metacognition  (e.g.,  Flavell,  1979;  Corno  &  Mandinach,  1983)  supports  the 
use  of  strategies  which  promote  comprehension  monitoring.  Schema  theory  (e.g.,  Anderson, 
Reynolds,  Schallert.  &  Goetz,  1977;  Norman,  Gentner,  &  Stevens,  1976;  Rumelhart  & 
Ortony,  1977)  leads  us  to  expect  that  journals  will  reflect  both  assimilation  and 
accommodation.  At  times  students  will  impose  schemata  on  new  information,  sometimes 
doing  "violence... to  the  data  contained  in  the  text"  (Anderson  et  al.,  1977,  371).  At  other 
times,  students  will  tune  schemata  or  create  new  cognitive  structures  (Norman,  Gentner,  & 
Stevens,  1976).  More  recent  perspectives  imply  that  journals  will  reflect  a  dynamic  process 
of  schema  assembly,  with  dynamic  networks  rather  than  stored  mental  structures  (e.g., 
Clancey,  1992;  Spiro,  Coulson,  Feltovich,  &  Anderson,  1988).  These  views  are  consonant 
with  theory  and  research  on  conceptual  change  (e.g.,  West  &  Pines,  1985;  Champagne, 
Gunstone,  &  Klopfer,  1985a  &  b;  Novak,  1985;  Strike  &  Posner,  1985;  Wittrock,  1985)  and 
with  constructivism,  with  its  emphasis  on  active  rather  than  passive  learners  (e.g.,  Duffy  & 
Jonassen,  1992;  Jonassen,  1991;  Wittrock,  1974). 

Unfortunately,  much  of  the  limited  research  on  the  use  of  journals  has  failed  to 
establish  meaningful  goals,  to  provide  challenging  tasks,  or  to  control  for  individual 
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difli'iviu-es.  For  example,  Anson  and  Bench  (1990)  used  length  of  journals  as  the  sole 
grading  criterion  and  Hynd  and  Chase  (1990)  failed  to  control  for  the  difficulty  level  of 
textual  materials. 

Purpose 

The  broad  purpose  of  this  predominantly  qualitative  study  was  to  construct  a  cognitive 
model  of  journal  writing  in  an  ecologically  valid  setting,  where  learner  characteristics  and 
task  difficulty  varied.  This  paper  focuses  on  three  of  the  research  questions  (for  discussion  of 
the  complete  study,  see  Cole,  1992). 

What  strategies  do  learners  use  when  they  are  encouraged  to  use  journals  as  a  cognitive 
tool  (Derry,  1990),  constructing  intermediate  stages  of  knowledge  (Bereiter  &  Scardamalia, 
1992)?  *       How  are  these  strategies  related  to  variables  such  as  content  difficulty,  reading 
ability,  and  cognitive  style?  For  example,  research  on  individual  differences  (Jonassen  & 
Grabowski,  1993)  implies  that  field  dependent  (FD)  students  will  want  more  direction  than 
field  independent  students  in  writing  journals  and  that  FD  journals  will  reflect  difficulty  in 
attending  to  details.  Miyake  and  Norman  (1979)  found  that  frequency  of  oral  questions 
about  text-editing  tasks  in  a  computer  program  (a  well-defined  domain)  was  .related  to 
expertise. 

Does  this  finding  extend  to  questions  and  comments  in  students'  journals  in  an 
Introduction  to  Literature  course,  which  is  an  ill-defined  domain  (Spiro  &  Jehng,  1990;  see 
also,  Spiro,  Coulson,  Feltovich,  &  Anderson,  1988)? 

Methods 

Setting  and  Subjects 

Qualitative  and  quantitative  methods  were  used  to  study  journal  writing  in  an  in  tact 
group  of  14  students  in  a  night  section  of  a  Introduction  to  Literature  at  a  large  urban 
community  college  in  the  midwest.  The  class  met  75  minutes  twice  a  week  for  16  weeks. 
The  basic  instructional  approach  combined  whole-group  discussion  with  elements  of  the 
cognitive-apprenticeship  model  (Collins,  Brown,  &  Newman,  1989).  On  the  first  night  of 
class,  I  told  students  that  I  would  he  collecting  information  that  would  help  me  improve 
instruction  for  them  as  well  as  for  future  students.  The  course  applied  toward  the 
humanities  requirement  for  the  AA/AS  degrees.  Course  objectives  emphasized  defining  and 
applying  literary  concepts,  applying  rules  and  heur-istics,  and  problem-solving.  During  the 
previous  seven  years,  I  had  used  similar  journal-writing  strategies  in  15-20  sections  of  that 
course,  as  well  as  in  other  courses.  None  of  the  students  had  previously  written  journals  of 
any  type.  Students  ranged  in  age  from  18  to  41  (mean  =  28  compared  to  the  college  mean  of 
31.2);  93%  worked  (67%  worked  full  time  or  more,  compared  to  the  college  mean  of  44%). 
The  average  credit  load  was  8.4  (compared  to  7.4).  All  of  the  students  planned  to  pursue 
baccalaureate  or  graduate  degrees  (compared  to  approximately  60%  in  the  college). 

Procedure 

The  Nelson-Denny  Test  of  Reading  Comprehension  (Form  Ef  1981  edition)  and  the 
Hidden  Figures  Test,  Part  1  (Ekstrom,  French,  &  Harman,  1976)  were  administered  at  the 
end  of  week  three.  Six  of  the  students  scored  at  or  above  the  16.9  grade  level  (the  highest 
level  on  the  exam)  on  all  three  measures:  comprehension,  vocabulary  and  total  reading.  Two 
read  below  grade  12.0  (11.8,  and  11.1),  with  comprehension  levels  of  12  and  9.4, 
respectively.  Two  students  were  very  field  dependent;  two  were  very  field  independent. 

Students  wrote  journals  on  five  stories  and  two  plays  (weeks  3  -  12).  Before  they  wrote 
their  first  graded  journal,  students  received  written  and  oral  instructions,  in -class  practice 
and  feedback,  three  written  descriptions  of  journal  writing  processes  by  former  students, 
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sample  journals,  and  encouragement  to  view  two  HyperCard  tutorials  on  journal  writing 
that  I  had  developed  specifically  for  the  course.*  All  instructions  encouraged  students  to 
write  any  question  or  comment  they  thought  would  help  them  understand  a  work  better.  As 
incentives,  the  journals  counted  22%  of  the  student's  course  grade;  grading  criteria  made  it 
difficult  for  students  to  earn  less  than  a  C  on  each  journal.  Following  are  the  instructions  in 
the  syllabus. 

Write  a  journal  for  each  story  and  play,  at  least  a  half-page  long — but  there  is  no 
maximum  length.  Write  your  journal  in  ink,  using  complete  sentences— otherwise  I  will  not 
be  able  to  understand  what  you  are  asking  or  saying.  Write  any  questions  you  have — except 
the  definition  of  a  word,  unless  you  are  unsure  of  the  relevant  definition — or  any  comments 
you  would  like  to  make  about  the  work.  The  journal  is  not  busy  work,  but  is  intended  to 
help  me  know  what  you  need  help  with  in  understanding  the  story.  Write  any  questic  is  you 
believe  will  help  you  understand  the  work  now  as  well  as  later,  for  the  examination — or  any 
other  questions  you  are  just  curious  about.  Journals  are  worth  a  maximum  of  10  points 
each.  If  a  journal  meets  the  minimum  requirements,  you  will  receive  7.5  points — a  grade  of 
C;  you  will  receive  additional  credit,  depending  on  how  much  your  questions  and  comments 
indicate  that  you  are  paying  attention  to  all  the  elements  as  you  try  to  understand  the  work. 
There  are  no  wrong  questions  or  comments  (but  there  are  questions  and  comments  which 
ignore  the  details  of  a  work).  Journals  are  a  tool  for  you  to  learn  about  what  you  don't 
know,  not  to  show  me  that  you  already  know  everything.  Even  if  you  don't  understand  a 
work  at  all  but  ask  questions  about  what  you  don't  understand  and  indicate  that  you  are 
paying  attention  to  details,  you  can  receive  a  10. 

I  will  collect  your  journals  at  the  beginning  of  the  class  on  which  they  are  due.  If  you 
miss  that  class,  you  may  submit  your  journal  at  the  next  class  you  attend. 

I  selected  and  sequenced  the  stories  and  plays  generally  in  order  of  increasing 
difficulty,^  using  feedback  I  had  received  from  students  over  the  years.  I  abandoned  my 
plan  to  rely  on  faculty  ratings  because  interrater  reliability  of  stories  five  faculty  were  all 
familiar  with  varied  from  -.76  to  .88.  I  also  rejected  ratings  from  the  Dale-Chall,  Fog, 
Flesch,  Flesch-Kincaid,  Fry,  and  Smog  reading  indices;  because  they  focus  on  factors  such  as 
sentence  length  and  the  number  of  syllables,  they  are  not  valid  indices  for  college-level 
stories  and  plays,  whose  difficulty  level  is  more  strongly  related  to  plot  structure,  historical 
context,  and  the  use  of  irony,  allusion  and  symbolism  (this  view  was  confirmed  by  two 
reading  teachers  at  the  college). 

To  establish  a  baseline  for  journal  writing  on  a  sufficiently  challenging  work,  the  first 
story  was  one  of  the  most  difficult.  To  examine  potential  differences  in  task  difficulty  after 
students  had  developed  some  expertise  in  writing  journals,  1  placed  the  easiest  story  fourth 
out  of  five. 

Data  were  collected  throughout  the  semester  in  journals,  questionnaires,  interviews  of 
five  students  selected  to  provide  insight  into  many  of  the  variables  of  interest  (age,  sex, 
reading  ability,  academic  major,  academic  record,  and  field  articulation),  standardized  tests, 
course  examinations,  and  field  notes  (recorded  after  each  class).  I  collected  journals  before 
class  discussion  of  each  work,  responded  in  writing,  made  a  copy  of  each  journal,  and 
returned  them  at  the  next  class.  Students  completed  a  written  questionnaire  about  journal 
writing  after  their  first  and  last  journals;  students  submitted  the  latter  to  my  supervisor, 
with  the  understanding  that  I  would  not  see  them  until  I  had  submitted  semester  grades. 

It  is  outside  the  scope  of  this  paper  to  describe  the  complete  study.  Major  variables  of 
interest  included  cognitive  activities;  reading  ability;  field  articulation;  story-difficulty  level; 
semester  GPA;  length  of  journals;  frequency  of  questions  and  comments;  topics  of  questions 
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and  comments;  self-ratings  of  amount  learned;  and  student  ratings  of  journal  helpfulness. 
Qualitative  analysis  focused  on  the  content  of  the  journals  on  the  stories  (particularly  the 
easiest  and  the  hardest  story),  interviews,  and  comments  in  class  and  on  questionnaires. 
Students'  journals  were  entered  in  a  data  base  and  each  main  clause  was  analyzed.  After 
attempts  to  apply  several  existing  taxonomies,  including  Bloom's  and  Gagne's,  I  developed  a 
taxonomy  intuitively  (see  Table  1).  Statistical  analyses  included  Pearson  product-moment 
correlations  and  ANOVAs. 

Results 

Following  are  the  most  salient  findings  on  the  major  research  questions.  (For  a 
complete  report,  see  Cole,  1992;  examples  of  students'  questions  and  comments  are  also  in 
Cole,  1993.) 

A  Cognitive  Model 

No  single  cognitive  model  could  capture  the  process  within  or  across  journals.  Journal 
writing  was  a  particularly  dynamic  activity,  which  students  adapted  to  emergent  needs.  It 
appeared  to  include  seven  major  components  (see  Fig.  1):  identifying  a  goal,  constructing  the 
text  base  and  constructing  a  situation  model  (Kintsch,  1989),  predicting  outcomes, 
identifying  significant  elements,  reflecting  on  meaning,  reconstructing  schema(ta).  As 
students  read  a  story  and  wrote  their  journals,  these  components  all  appeared  to  interact 
with  each  other  as  well  as  with  other  factors,  such  as  individual  learner  characteristics  and 
task  difficulty  (see  Fig.  2  and  Table  2). 

Students  engaged  in  a  variety  of  preparation  and  execution  strategies.  For  example,  all 
hut  one  generally  read  a  story  more  than  once  (up  to  10  times);  some  read  a  story  initially 
just  to  get  "a  feeling  for  the  flow/'  Some  stopped  immediately  to  look  up  unknown  words; 
some  preferred  to  infer  meanings.  Some  used  the  "Catechism  for  Stories,"  a  set  of  questions 
on  setting,  conflict,  etc. ,3  almost  as  a  recipe;  some,  as  a  heuristic;  some  never  read  it.  As 
students  gained  experience 
Table  1.  Taxonomy  of  Students'  •Journal  Entries 

1.  Correct — i.e.,  a  tenable  inference/explnnntion 

2.  Incorrect — clearly  contradicted  by  evidence  from  the  story 

3.  Neither  Correct  nor  Incorrect 

4 .  Statement  of  Confusion — explicit 

5.  Question 

6.  Direct  Open  Question 

An  interrogatory  sentence  that  can't  ho  answered  "yes"  or  "no." 

Example:      Why  doesn't  the  author  give  us  the  names  of  the  husband  and  wife? 

7 .  Direct  Closed  Question 

An  interrogatory  sentence  that  enn  hp  nriHwcrod  "yen"  or  "no." 
Example:      Are  they  talking  about  an  abortion? 

8.  Indirect  Open  Question 

An  implied  question  which  can't  be  answered  "yes"  or  "no." 
Example:      I  wonder  why  he  was  in  this  country  to  begin  with. 
[Why  was  he  in  this  country  to  begin  with?] 

9.  Indirect  Closed  Question 

An  implied  question  which  can  be  answered  "yes"  or  "no." 
Example:      I  think  they  are  talking  about  an  abortion  but  I'm  not  sure. 
[Are  they  talking  about  an  abortion?] 

10.  Item  Supporting  a  Question 
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1 1 .  Reviewing  Content  (Summarizing/Paraphrasing) 

12.  Self-explanation 

13.  Item  Supporting  a  Self-explanation 

14.  World  Knowledge — e.g.,  "The  story  takes  place  in  Spain  because  it  mentions  Madrid 
and  Barcelona  [which  are  in  Spain].1'  In  a  looser  sense,  many  of  the  self-explanations  drew 
upon  declarative  world  knowledge;  for  example,  "The  man  mentions  that  he  loves  the 
woman  he  is  with,  but  the  woman  is  never  referred  to  as  his  wife.  So  I  am  going  to  assume 
they  are  not  married  [because  descriptions  of  married  people  use  terms  such  as  husband 
and  wife]? 

15.  Character 

16.  Says  or  does 

17.  Thinks 

18.  Feels 

19.  Has  a  Given  Trait 

20.  Setting — explicit  use  of  the  term 

2 1 .  Point  of  View — explicit  use  of  the  term 

22 .  Theme/Subject — explicit  use  of  the  term 

23.  Protagonist/Antagonist— explicit  use  of  the  term 

24.  Plot  Element — explicit  use  of  the  terms  conflict,  crisis,  climax,  denouement 
(These  were  grouped  becuuse  there  were  so  few  instances  of  each.) 

25.  Irony — explicit  uho  of  tho  term 

26.  Other  Elementu  (e.g.,  foreshadowing,  imagery,  plausibility,  symbolism,  title) 

27.  Judgment 

28.  Character's  Actions 

29.  Author's  Actions  or  Story  in  General 

30.  What  the  author  did— explicit  reference  to  author 

3 1 .  Why  the  author  did  what  he/she  did — explicit  reference  to  author 

32 .  Story  Difficulty — explicit  reference  to  difficulty 
3  3 .  Statement  of  Affect 

34.  Other  (items  not  classifiable;  e.g.,  statements  about  the  process  or  the  student's  life) 

35,  Story-specific  Items 
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Table  2.  Subdivision  of  Factors  Which  Influence  Journal  Writing 

I.  Overt  Activities 
A.    Head  story. 

It.    Write  in  journal. 

I I .  Strutetfies/Acti  vities 

A.    Metacognitive  Strategies/Activities 

1.  Plan. 

2.  Kxecuto. 

H.    Cognitive  Strategies/Activities 

1 .  Review  content. 

2.  Problem  solve. 

a.  Ask  questions. 

b.  Generate  self-explanations. 

3.  Evaluate. 

III.  Individual  Learner  Characteristics 
A.  Aptitude 

1.  Reading  ability 

a.  Comprehension 

b.  Vocabulary  level 
(\     Reading  speed 

2.  Domain  knowledge 

.'i.    (ionernl  world  knowledge 

4.  Metucngnitive/C  Cognitive  Ability 
H.    Memory  Capacity 

C     Field  dependence/independence 
I).  Motivation 

1 ,    Value  of  the  activity  to  the  student 

2     Student's  confidence  in  ability  to  write  journals 

H.  Self- appraisal  of  ability  to  understand  stories 

K.    Understanding  of  the  task  (also  related  to  motivation) 

IV.  Task  Difficulty 

A.  Frequency  of  unknown  words  in  the  story  (from  none  to  many) 

B.  Use  of  conventions  of  grammar  and  of  fiction  (generally  uses  or  disregards) 

C.  Amount  of  irony  in  the  story  (from  none  to  much) 

D.  Clarify  of  the  situation  model  (see  Kintsch,  1989) 

E.  Understanding  of  journal  writing 

I .  Purpose 
2 .  Criteria 

F .  Time  available  to  complete  task 
V.    External  Resources 

A.    Reference  works 

1 .  Dictionaries 

2.  Encyclopedias 

3.  Atlases 

B.  Literary  criticism 

1 .  Articles 

2.  Books 

C.  Other  people 

1 .  Fellow  students 

2.  Family 

IJ ,    Friends/cowor  kers 
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VI    Teacher  Kxpectations  (Criteria  for  journal) 


In  imirmil  writing,  I  hoy  tfuiclcul  to  rely  less  on  the  "Catechism"  as  an  external  guide  and 
instead  In  be  guided  by  their  internal  needs.  Some  said  story  difficulty  influenced  their 
sli'iiti'i'ios;  tor  example,  the  best  journal  writer  said  she  usually  took  notes  as  she  read  but 
I  hut  "Hills  Like  White  Klephants"  was  so  confusing  she  read  it  three  times  before  she  began 
t uk me;  notes,  unci  then  rend  it  seven  more  times.  However,  most  students  generally 
activated  a  single  procedure  that  was  already  in  their  repertoires,  This  seemed  particularly 
t  rue  of  the  more  field  dependent  students,  who  relied  on  external  support  in  understanding 
the  stories;  they  generally  used  a  discussion  strategy  (asked  friends  or  family  what  they 
thought  the  stories  meant  before  they  wrote  their  journals),  or  used  the  "Catechism"  or 
adjunct  questions  to  guide  their  journal  writing. 

In  spite  of  repeated  attempts,  it  proved  impossible  to  identify  the  depth  of  cognitive  pro- 
cessing, because  journals  are  only  cognitive  artifacts,  not  records  of  students'  thought 
processes.  For  example,  in  an  interview,  the  most  astute  reader  said  he  didn't  really  predict 
outcomes,  but  automatically  ran  several  possible  scenarios  through  his  mind  as  he  read, 
rejecting  scenarios  as  new  details  made  them  untenable;  yet  ho  did  not  describe  these 
activities  (or  any  of  the  rejected  scenarios)  in  his  journals.  And  if  a  student  wrote  "What 
difference  would  it  make  if  the  setting  were  changed?,"  it  was  impossible  to  determine  if  the 
student  was  mindlessly  copying  a  question  from  the  "Catechism"  or  sincerely  trying  to 
problem  solve. 

The  use  of  schemata  in  constructing  meaning.  The  content  of  students1  journals 
suggested  that  they  attempted  to  construct  meaning  by  activating  relevant  schemata, 
refining  their  existing  schemata,  and/or  creating  new  schemata,  as  suggested  by  prior 
research  (Anderson,  1977;  Rumelhart  &  Ortony,  1977).  In  terms  of  this  theory,  when 
existing  schemata  readily  accommodated  story  elements,  students  would  not  experience  any 
cognitive  dissonance  (Osborne  &  Wittrock,  1983;  Wilson  &  Cole,  1991)  and  therefore  would 
not  usually  mention  those  elements  in  their  journals.  However,  data  from  interviews  and 
journals  suggest  that  when  students'  existing  schemata  did  not  readily  accommodate  story 
elements,  most  students  summarized,  generated  self-explanations,  or  asked  questions. 
Furthermore,  when  students  experienced  too  much  cognitive  dissonance  (e.g.,  in  trying  to 
understand  "Hills  Like  White  Elephants"),  they  resorted  to  stating  their  confusion  in  general 
terms  or  blaming  the  author. 

Assimilation  theory  (Mayer,  1975,  1979,  1980)  provides  additional  insight  into  the  stu- 
dents' use  of  schemata  in  problem  solving.  The  theory  suggests  that  meaningful  learning 
(or,  in  this  case,  understanding)  depends  on  three  conditions:  reception  of  material,  and 
availability  and  activation  of  an  assimilative  context  (Mayer,  1980).  However,  students  did 
not  always  receive  the  material;  they  missed  details.  Nor  did  students  always  have  an 
assimilative  context.  This  was  particularly  evident  in  their  journals  on  the  most  difficult 
stories.  For  example,  most  students  did  not  have  a  context  for  assimilating  the  "operation" 
in  "Hills  Like  White  Elephants,"  an  outdated  mode  of  abortion  which  is  described  in  the 
story  as  "letting  the  air  in."  Some  students  mainly  asked  questions  or  expressed  confusion 
instead  of  trying  to  generate  explanations  (e.g.,  "What  type  of  operation  is  the  girl  going  to 
have?"  and  "The  sentence  'It's  just  to  let  air  in'  I  do  not  understand  at  all!!'1— student's 
punctuation).  Generally,  however,  the  journal  questions  and  comments  did  provide  evidence 
that  students  were  trying  to  use  an  assimilative  context.  In  "Hills  Like  White*  Klephants," 
one  field  dependent  reader  initially  missed  the  repeated  use  of  the  word  "operation."  In  a 
subsequent  reading,  having  noticed  the  word,  he  inferred  thnt  "operation"  referred  to  the 
business  operation  of  the  bar  at  the  train  station   Later,  when  he  was  apparently  eucd  hy 
other  students'  imposition  of  a  false  assimilative  context  during  discussion  before  class,  he 


ERIC 


decided  thi1  operation  was  to  restore  the  girl's  hearing.  By  contrast,  the  best  reader 
generated  the  following  explanation:  "The  only  thing  that  I  can  think  of  that  would  not 
really  he  un  operation,  that  would  relieve  the  tension  or  unhappiness  that  they  both 
apparently  feel,  would  he  an  abortion." 

When  readers  aetivate  schemata,  they  run  the  risk  of  imposing  inappropriate  schemata 
(Anderson  et  al.  1  977).  This  was  particularly  evident  in  the  journal  on  "Hills  Like  White 
Klcphnnts."  Apparently  unable  to  make  sense  of  some  key  details,  several  students 
attended  to  selective  details  that  fit  a  schema  they  could  understand;  for  example,  because 
the  girl  could  not  understand  the  waitress  and  asked  the  man  to  tell  her  what  the  waitress 
said,  several  students  concluded  the  girl  was  deaf  (see  related  example  in  paragraph  above). 
In  a  slightly  different  vein,  all  the  students  activated  an  incomplete  schema  for  "white 
elephants."  None  of  them  had  additional  schemata  for  a  white  elephant  as  something 
precious  which  is  difficult  and  expensive  to  maintain  or  an  endeavor  that  is  a  failure. 

How  are  these  activities  related  to  individual  learner  characteristics?  The  greater  the 
students'  expertise  in  reading,  the  less  they  tended  to  review  content  (summarize)  in  the 
easiest  story.  This  would  be  expected  if  the  purpose  of  summarizing/paraphrasing  is  to  help 
students  monitor  their  comprehension  (Palincsar  &  Brown,  1989),  However,  on  the  more 
difficult  stories,  some  of  the  K%\t  readers  used  summarizing  to  scaffold  themselves  in 
developing  insights. 

Students'  cognitive  activities  were  not  significantly  related  to  students'  ratings  (a)  of 
journal  helpfulness  (at  the  beginning  or  end  of  term),  (b)  of  how  much  they  said  they  had 
learned  about  understanding  literature,  or  (c)  of  the  "Student  Accounts  of  Journal  Writing" 
(an  instructional  resource). 

Although  the  lower  achievers  (as  measured  by  semester  GPA)  tended  to  write  as  much 
as  the  higher  achievers,  they  tended  to  write  more  off-task  statements,  for  example,  affective 
statements  or  statements  about  the  subject  or  author.  (The  issue  is  not  whether  students 
engage  in  off-task  behavior,  but  the  extent  to  which  that  supplants  their  engaging  in 
activities  that  support  their  achieving  the  stated  goals  of  the  assignment  and  the  course.) 

Types  of  Questions  and  Comments 

An  analysis  focusing  on  the  hardest  and  the  easiest  story  revealed  that  students 
generally  engaged  in  three  types  of  cognitive  activities  in  their  journals:  reviewing  content 
(summarizing  and  paraphrasing),  evaluating  (making  judgments  about  character  actions, 
author  actions,  or  the  story),  and  problem  solving  (asking  questions  and  generating  self- 
explanations).  The  elaborated  classification  is  reminiscent  of  the  strategies  "good  students 
routinely  bring  to  the  task  of  studying  texts":  questioning,  clarifying,  summarizing,  and 
predicting  (Brown  &  Palincsar,  1989,  p.  414).  The  classification  is  also  suggestive  of  Chan, 
Hurt  is,  Scardamalia,  and  Boreiter's  (1992)  declarative,  interrogatory,  and  evaluative 
"response  subtypes"  (p.  103).  However,  those  two  classification  schemes  relate  to  what  Ng 
and  Hereitor  (1991)  called  "extracting  knowledge  from  text,"  while  constructing  meaning  from 
short  storirs  requires  interpretation  more  than  extraction. 

Thorc  were  significantly  more  self- expla tint  ions  (comments)  than  questions  in  each  of 
journals  2  through  ft  (Tables  l\  and  4).  The  Hinnlnr  frequencies  in  the  first  story  may  be 
related  to  n  combination  of  fat-tors:  its  brevity  (1 70  words),  its  difficulty,  and  students' 
inexperience  at  writing  journals.  Small  to-moderate  positive  correlations  (Table  ft)  between 
the  frequency  of  self  explanations  in  pairs  of  journals  suggests  that  certain  students 
habitually  engaged  in  generating  (many  or  few)  sell  explanat  ions,  regardless  of  story 
difficulty.  However,  there  were  no  put  terns  in  frequency  of  questioning  and  of  reviewing 
content.  There  was  a  small  significant  relationship  between  question  frequency  (r  =  .613,  p 


1  10 


ERIC 


<  .05)  in  journals  4  and  5,  suggesting  that  some  students  might  have  been  developing  a 
habit  of  generating  or  not  generating  questions. 

Focus  of  cognitive  activities.  Preliminary  analyses  indicated  that  students'  cognitive  ac- 
tivities focused  on  three  categories:  (a)  the  textbase  (Kintsch,  1989);  (b)  the  situation  model 
(Kintsch,  1989),  which  in  a  literary  work  involves  elements  such  as  setting;  characters* 
actions,  thoughts,  and  feelings;  conflict;  outcome;  and  subject,  gist,  and  theme;  and  (c) 
literary  principles,  which  might  be  called  the  aesthetic  model  (e.g.,  why  the  author  included 
a  particular  character,  and  what  would  be  gained  or  lost  by  changing  the  point  of  view  or 
setting).  The  majority  of  cognitive  artifacts  were  attempts  to  understand  the  situation 
model,  even  in  the  journals  on  the  easiest  story.  Students  admitted  confusion,  asked 
questions,  generated  self-expla  nations,  judged  characters'  and  authors'  actions,  and 
discussed  elements  that  teachers  might  not  point  out  in  class;  some  also  responded 
affectively.  The  findings  are  generally  consonant  with  Wilson's  (1989)  description  of  journals 
in  an  1 1th  grade  high  school  literature  class,  Most  of  the  students*  questions  and  comments 
were  directly  related  to  the  stories,  particularly  elements  relating  to  character.  Students 
wrote  significantly  more  (p  <  .Of))  about  the  story  elements  (e.g.,  conflict,  crisis,  climax, 
theme)  in  the  eesiest  than  in  the  hardest  story. 


Table  A,  Mean  Numbers  of  Cognitive  Activities  in  Journals  on  Stories 


Journal 

Problem 

Self-explanations 

Solving 
Questions 

Content  Reviews 

1  hard* 

3.58  (3.87)** 

3.58  (3.45) 

1.08 

2  easy 

10.75a  (11.44) 

1.83a  (2.17) 

.25 

3 

hardest 

10.58b  (9.56) 

2.83b  (3.76) 

3.17  (2.67) 

4  easiest 

12.92c  (12.56) 

2.33c  (4.03) 

6.83  (8.21) 

5  hard 

14.25d  (9.49) 

3.25d  (2.83) 

8.58  (6.57) 

n  =  12  (Only  students  who  completed  all  5  journals  were  included.) 
*  story  difficulty 

**  Standard  deviations  are  in  parentheses.) 
af  b,  c,  d:  p  <  .05 


Table  4.  Effect  of  Story  Difficulty  on  Average  Number  of  Items  in  Journals  2  -  5* 


Questions 

Self-explanations 

Reading  Level 

Easy  Stories 

Hard  Stories 

Easy  Stories 

Hard  Stories 

High  Ability  (6) 

5.00 

6.50 

23.17 

26.83 

Lower  Ability  (6) 

3.33 

5.67 

24.17 

22.83 

Total  (12) 

10.25a 

48.50a 

Std.  Dev. 

9.54 

38.14 

*  For  this  analysis  journal  1  was  treated  as  practice. 
pa  <  .001 
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Table  5.  Correlations  of  Numbers  of  Self-Explanations  Between  Journals 


Journal 

2 

3 

4 

5 

1 

.868a 

.806a 

.671b 

.638b 

2 

.742b 

.576b 

.649b 

3 

.944a 

.666b 

4 

.693b 

n  =  12  (Only  students  who  completed  all  5  journals  were  included.) 
pa  <  .01  pb  <  .05 


The  role  of  difficulty  level.  Story  difficulty  level  was  significantly  related  to  only  two 
journal  variables:  journal  length  and  number  of  story  elementH  addressed.  Students  wrote 
significantly  more  for  the  easiest  story  (fourth)  than  the  hardest  story  (third).  However,  this 
difference  might  be  explained  by  the  fact  that  students  tended  to  write  more  as  they  gained 
more  experience;  the  average  length  of  the  five  journals  on  stories  increased  steadily:  182.8 
(std,  dev.  128.2)  ,  349.7  (328.2),  398.8  (325.2),  539.9  (511.4),  644.5  (327.1).  An  ANOVA 
and  post  hoc  analyses  revealed  significant  (p  <  .05)  differences  between  journals  1  and  5 
and  1  and  4.  More  important  than  the  difference  in  length  is  the  fact  that  students 
addressed  significantly  more  story  elements  (particularly  classical  plot  elements)  in  their 
journals  on  the  easiest  story  than  on  the  hardest.  This  difference  suggests  a  hierarchical 
relationship,  with  construction  of  the  situation  model  being  requisite  to  identifying  all  but 
the  simplest  plot  elements;  because  most  students  could  not  construct  the  situation  model 
in  the  hardest  story,  the  only  plot  element  they  tried  to  identify  was  the  setting. 

Miyake  and  Norman  (1979)  found  that  on  easier  material  novices  asked  more 
questions,  but  on  harder  material  trained  subjects  asked  more  questions  than  novices.  In 
the  current  study,  because  most  of  the  cognitive  artifacts  in  the  journals  were  attempts  to 
understand  the  situation  model.  I  decided  to  examine  the  frequency  of  questions  and  self- 
explanations  in  terms  of  reading  expertise  rather  than  knowledge  of  literary  principles. 
Using  reading  grade  level  as  an  index  of  expertise,  analysis  did  not  support  these 
hypotheses  (Table  4).  This  might  be  explained  by  the  fact  that  students  in  the  high-ability 
group  all  read  at  or  above  the  16.9  grade  level,  but  only  one  or  two  students  in  the  lower- 
ability  group  might  truly  be  described  as  low-ability,  and  none  could  be  described  as  novice 
readers. 

However,  sevL  al  differences  between  this  study  and  the  Miyake-Norman  study  are 
worth  noting.  One  of  the  most  important  differences  is  the  procedure  for  identifying 
questions.  In  the  earlier  study,  students  were  asked  to  verbalize  their  questions  aloud  as 
they  studied  instructional  materials,  while  in  this  study  students  wrote  their  questions  and 
comments,  often  over  the  course  of  several  hours  or  even  days;  they  answered  many  of  their 
questions  before  they  began  to  write  their  journals.  Second,  "prior  knowledge"  proved  to  be 
a  much  fuzzier  term  in  this  study  than  in  the  Miyake-Norman  study,  where  students  had  a 
common  goal  in  a  well-defined  domain  (learning  three  text-editing  tasks);  thus  relevant  prior 
knowledge  was  clearly  defined.  In  the  current  study,  students  were  encouraged  to  adapt  a 
very  generalized  task  to  their  own  needs  in  an  ill-defined  domain.  Although  the  long-term 
goal  was  to  help  students  develop  their  skills  in  understanding  stories,  they  were 
encouraged  to  use  the  journal  to  address  their  immediate  needs  (as  thvy  defined  them)  in 
understanding  a  given  story.  Thus,  the  immediate  goal  (and  perceived  relevance  of  prior 
knowledge)  differed  for  each  student. 
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Third,  several  types  of  prior  knowledge  played  a  role  iu  this  study,  from  prior 
conceptions  of  journal  writing  to  general  world  knowledge.  1  measured  students'  reading 
abilities,  but  I  did  not.  attempt  to  formally  measure  several  other  types  of  prior  knowledge, 
such  as  students'  prior  knowledge  of  fiction  (declarative  and  procedural).  Perhaps  the  most 
critical  prior  knowledge  was  students'  general  world  knowledge,  which  Anderson  et  al. 
(1977)  and  Hrown,  Campione,  and  Day  (1981)  suggested  may  be  one  of  the  most  important 
factors  in  students'  abilities  to  comprehend  expository  prose.  General  world  knowledge 
would  logically  Boom  to  play  an  even  greater  role  in  comprehending  college-level  literature. 
But  there  was  no  way  to  measure  all  the  general  knowledge  that  directly  or  indirectly  had 
bearing  on  the  various  stories  students  read.  Nor  was  there  a  way  to  identify  all  the 
knowledge  that  might  have  interfered  with  understanding  a  story,  with  students' 
assimilating  information  into  inappropriate  schemata. 

Three  additional  points  are  worth  noting  about  the  relationship  between  story  difficulty 
and  questions.  First,  if  the  point  of  interest  is  not  quantity  (number  of  questions)  but 
quality  (the  type  of  elements  addressed  in  questions  and  comments),  then  this  study  does 
provide  some  evidence  of  a  relationship.  Analysis  of  journal  content  revealed  two  qualitative 
differences.  Compared  to  lower-ability  readers,  the  high-ability  readers  tended  to  generate 
higher-level  inferences  (e.g.,  inferring  that  the  operation  in  "Hills  Like  White  Elephants"  was 
an  abortion,  and  commenting  on  the  role  of  the  setting  vs.  the  setting  itself),  and  they  tended 
to  address  their  own  questions  and  points  of  confusion.  Future  research  should  investigate 
whether  these  tendencies  are  related  to  other  factors,  such  as  confidence,  willingness  to  take 
risks,  or  knowledge  of  reading  strategies. 

Second,  students1  goal  orientations  (Ng  &  Bereitcr,  1001)  may  have  been  related  to  the 
frequency  of  their  questions  and  self-explanations.  This  possibility  warrants  future 
research,  particularly  since  total  student  load  (academic  plus  work)  correlated  negatively 
with  length  of  journals  (e.g.,  r  =  -.681,  -,676,  and  -.KOI  for  the  last  three  story  journals;  p  =  < 
.02),  although  average  length  of  journals  increased  steadily.  Third,  the  higher  frequency  of 
self -explanations  than  questions  suggests  that  they  deserve  equal  attention  and  that 
research  in  ecologically  valid  contexts  should  not.  restrict  students  to  one  or  the  other. 

How  arc  questions  and  comments  related  tu  individual.lmuimr  charqcturiatiL^  Field 

articulation  and  reading  ability  were  the  major  learner  characteristics  of  interest.  As  the 
literature  predicted  (Jonassen  &  Orahowski,  10970.  then!  was  n  significant  relationship 
between  scores  on  the  Hidden  Figures  Test  and  reading  comprehension,  vocabulary  and 
total  reading  levels  (r  =  .68,  .59  and  .57,  respectively,  with  p  =  <  .05).  A  relationship 
between  reading  ability  also  correlated  strongly  with  semester  UPA  (e.g.,  total  reading  level 
r  =  .HI,  p  =  .001).  The  reason  for  the  latter  must  bo  further  explored.  Other  correlations 
suggest  two  possibilities,  both  of  which  involve  metacognitive  skills  of  planning  and 
execution.  The  first  involves  the  student's  carrying  a  load  (academic  and  work)  too  heavy  to 
master  with  his/her  reading  ability  (and  perhaps  study  skills,  etc.).  The  second  involves 
engaging  in  off-task  behavior  that  supplanted  behavior  that  was  required  for  mastering 
objectives  (see  correlations  in  research  question  2). 

Neither  reading  ability  nor  field  articulation  were  significantly  related  to  any  other  ma 
jor  factor.  Failure  to  obtain  statistically  significant  relationships  may  have  been  a  function 
of  the  small  sample  sizes  (less  than  six  field  dependent  and  field  independent  students). 
However,  tendencies  consonant  with  prior  research  on  field  dependence  and  field 
independence  (Jonassen  &  Orahowski,  109*0  suggest  the  need  to  explore  several  issues  with 
larger  sample  sizes,  particularly  since  the  ability  to  perceive  parts  of  a  whole  is  essential  iu 
the  analytical  and  critical  thinking  required  in  the  course.  Analysis  of  journal  content  and 
in  class  comments  suggested  that  Held  dependent  students  were  less  likely  than  field 
independent  students  to  perceive  key  elements/significant  details,  particularly  as  story 
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difficulty  increased.  Moreover,  when  they  were  confused,  field  dependent  students  often 
attended  to  superficial  details  and  meanings,  while  field  independent  students  tended  to 
utilize  details  to  help  them  generate  self-explanations.  Reflecting  a  reliance  on  external 
guidance  and/or  a  preference  for  social  contexts,  the  field  dependent  students  also  tended  to 
rely  on  the  "Catechism"  (and  the  adjunct  questions  included  in  the  text  for  one  story)  and/pr 
rely  on  interpretations  of  family  and  friends.  In  contrast,  field  independent  students  seemed 
to  attend  to  internal  cues,  which  allowed  them  to  address  their  own  needs;  they  either 
ignored  the  "Catechism"  or  used  it  just  "to  get  my  wheels  rolling." 

Two  field  dependent  students  were  classic  examples  of  the  inability  to  perceive  details. 
For  example,  in  his  journal  on  "Hills  Like  White  Elephants,"  Todd  said  he  initially  missed 
the  word  operation,  even  though  it  was  used  several  times  and  was  the  focus  of  the  story. 
After  he  finally  noticed  the  word,  he  missed  other  details  that  would  have  restricted  its 
meaning,  and  instead  thought  that  it  referred  to  the  business  operation  of  the  bar.  His 
journal  comments  suggest  that  a  discussion  before  class  made  him  realize  that  the  word 
referred  to  a  medical  operation,  but  he  still  failed  to  identify  significant  elements  of  the  story 
that  would  have  helped  him  correctly  identify  the  operation  asan  abortion.  By  contrast,  he 
appeared  to  be  guided  by  adjunct  questions  in  his  journal  on  the  previous  story;  he 
addressed  three  significant  issues,  all  in  the  order  they  appeared  in  adjunct  questions. 
Taxonomy 

A  taxonomy  (Table  1)  can  describe  students'  questions  and  comments  only  in  very 
general  ways  in  this  ill-defined  domain.  Having  to  work  with  cognitive  artifacts  made  it 
impossihle  to  determine  levels  of  cognitive  processing  and  to  confidently  classify  all  the 
entries  in  journals.  Students  generally  did  not  address  the  critical  elements  that  are  the 
focus  of  most  instruction  and  critical  analyses.  Instead,  content  of  journals  reflected 
intermediate  stages  of  knowledge  construction  (Bereiter  &  Scardamalia,  1992)  which  was 
highly  idiosyncratic.  The  idiosyncrasy  within  and  across  sets  of  journals  is  consonant  with 
chaos  theory:  The  sensitive  dependence  on  initial  conditions — e.g.,  learner  knowledge — can 
result  in  large,  unexpected  effects — e.g.,  the  specific  questions  and  comments — (Gleick, 
1987). 

Student  Perceptions  of  Journal  Writing 

Twelve  students  evaluated  journal  writing  during  the  third  and  thirteenth  weeks  of  the 
sonu'stcr,  knowing  that  I  would  not  sec  their  comments  until  I  had  submitted  course*  grades. 
High  ratings  (mean  -  M.9  out  of  f)  on  each;  std,  dev.  =  .99f>  and  1.17)  were  supported  by 
responses  to  open-ended  quest  ions  on  the  survey,  interviews,  class  comments,  and  analysis 
of  journal  content.  In  the  second  evaluation,  nine  students  described  the  most  helpful 
aspect  of  journal  writing  in  terms  of  a  cognitive  tool  (I)erry,  1990;  Jonassen,  1992).  They 
said  it  (a)  stimulated/facilitated  their  thinking,  including  articulating  or  clarifying  thoughts, 
formulating  opinions,  examining  all  of  a  story's  points;  (h)  forced  them  to  think  about  what 
they  rend,  instead  of  reading  passively  or  just  for  pleasure;  (c)  made  them  look  at  a  work  in 
more  than  one  light;  (d)  taught  them  to  concentrate1  or  focus  thoughts;  and  (e)  forced  them  to 
read  more  carefully,  including  looking  for  details.  Yet  some  of  the  students  who  rated 
journal  writing  mosl  highly  said  they  would  not  have  done  it  if  it  had  not  heen  required. 

How  i\u  ratings  of  journal  helpfulness  relate  to  other  variables?  Ratings  were  not 
significantly  related  to  reading  ability  level,  field  articulation,  average  grades  on  journals,  or 
grades  in  the  course  at  the  time  of  the  survey.  Initial  ratings  of  journal  helpfulness  had  a 
moderate  positive  correlation  with  final  ratings  (r  =  .699,  p  =  .01),  and  a  moderate  negative 
correlation  with  self-ratings  of  skill  in  understanding  stories  at  the  beginning  of  the  term  (r  = 
•.7()6,  p  =.01). 4  At  the  end  of  the  term  the  correlation  between  journal  helpfulness  and  skill 
in  understanding  stories  had  decreased  slightly  (r  =  -.657,  p  =  .02).  The  findings  suggest  the 
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importance  in  quickly  identifying  students  who  rate  their  understanding  of  stories  highly 
and  helping  them  discover  how  to  use  journal  writing  as  a  cognitive  tool. 

Several  other  factors  significantly  correlated  with  students'  ratings  of  journal 
helpfulness  at  the  end  of  the  term  (see  Table  6).  For  example,  the  more  they  thought  they 
had  learned,  the  higher  they  rated  journal  writing.  The  fact  that  neither  journal  grades  nor 
course  grades  correlated  significantly  with  ratings  of  helpfulness  may  suggest  that  students 
were  learning  strategies  and  skills  that  examinations  were  not  sensitive  to  (e.g., 
development  of  metacognitive  strategies).  Since  no  attempt  was  made  to  measure  learning 
gains  in  terms  of  the  material  on  the  examinations,  it  is  possible  that  students  who  thought 
they  had  learned  a  lot  had  indeed  done  so. 

Table  6.  Factors  Correlated  with  Ratings  of  Journal  Helpfulness  After  Last  Journal 


Factor    r  p 

1 .  self-rating  of  how  much  was  learned  about  reading  literature    .902       .00 1 

2.  rating  of  helpfulness  of  journal  writing,  after  first  journal   .699  .01 

3.  academic  load      -.680  .02 

4 .  self-rating  of  skill  in  understanding  stories,  beginning  of  term   -.657  .02 

5.  improvement  in  confidence  in  ability  to  write  journals,  last  day  of  term  .640  .02 

6.  rating  of  helpfulness  of  Student  Accounts  of  Journal  Writing     ,594  ,05 

7.  semester  GPA      .588  .05 
Summary 

Findings  indicated  that  (a)  students  generally  viewed  journal  writing  as  a  flexible  cogni- 
tive tool  which  helped  them  construct  the  meaning  of  stories,  and  (b)  writing  journals 
scaffolds  students  in  attending  to  details,  asking  questions,  and  answering  their  own 
questions.  Moreover,  journal  writing  is  a  very  dynamic  process;  no  single  cognitive  model  can 
describe  journal  writing  either  within  or  across  students.  A  two-part  model  was  developed 
to  describe  the  process:  (a)  factors  which  influence  journal  writing  (task  difficulty,  individual 
learner  characteristics,  teacher  expectations,  student  strategies,  external  resources,  overt 
activities)!  and  (b)  components  of  journal  writing  (establishing  a  goal,  constructing  the 
texthase,  constructing  the  situation  model,  predicting  outcomes,  identifying  significant 
elements,  reflecting  on  meaning,  and  assembling  the  schema).  Although  high-ability  readers 
tended  to  engage  in  different  cognitive  activities  than  lower-ability  readers,  frequencies  of 
questions  and  self-explanations  were  not  related  to  story  difficulty.  Students'  valuations  of 
journal  writing  were  not  significantly  related  to  measures  of  achievement  (course  grades, 
semester  grade-point  averages,  or  cumulative  grade-point  averages). 


Discussion 


A  Flexible  Tool 

The  data  from  surveys,  journals,  interviews,  and  comments  suggest  that  journal  writing 
can  be  a  flexible  cognitive  tool  for  a  wide  range  of  students  in  a  college  Introduction  to 
Literature  class.  The  fact  that  students  of  diverse  ability  and  achievement  rated  journal 
writing  as  helpful  supports  empirical  data  of  Chi,  de  Leeuw,  Chiu,  and  LaVancher  (1991) 
that  high-ability  and  average-ability  students  benefit  equally  from  being  prompted  to 
generate  self-explanations  in  physics  classes. 

Although  most  students  said  they  found  journal  writing  helpful,  even  students  who  said 
they  "enjoyed"  journal  writing  did  not  write  optional  journals.  This  might  reflect  the  fact 
that  they  had  not  had  enough  experience  with  journal  writing  to  incorporate  it  into  their 
repertoire  of  study  strategies;  Duffy  and  Roehler  (1989)  found  that  high  school  students 
needed  at  least  six  months  of  frequent  practice  with  a  new  strategy.  Here,  the  new  strategy 
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had  to  compete  not  only  with  presumably  highly  automatized  strategies,  but  also  with 
personality  factors  (such  as  self-described  "laziness")  and  demands  of  work,  other  courses, 
and  spouse/family.  Furthermore,  students  may  have  needed  guidance  in  applying  journal 
writing  to  the  study  of  poetry,  which  most  students  find  intimidating  on  its  own.  Several 
people  have  argued  that  each  literary  genre  requires  knowledge  of  its  own  conventions  (e.g., 
Culler,  1975;  Pressley,  Goodchild,  Fleet,  Zajchowski,  &  Evans,  1989). 

Task  orientation.  Some  students  may  have  had  task-completion  rather  than  personal 
knowledge-building  orientations  (Ng  &  Bereiter,  1991).  This  seems  particularly  likely  for  the 
student  who  rated  journal  writing  "Not  at  nil  Helpful."  (Note:  This  student  was  not 
included  in  many  analyses  because  she  missed  class  the  night  students  took  the  reading 
and  HFT  tests  and  refused  to  make  thum  up.)  Helping  students  become  "intentional" 
learners  is  likely  to  require  more  than  strategy  instruction  (Bereiter  &  Scardamalia,  1989), 
particularly  for  adult  learners,  who  have  such  highly  developed  schemata  for  school  learning. 

Self-explanations  and  scaffolding.  One  explanation  of  the  mechanism  of  self- 
explanations  is  provided  by  Chi  and  VanLehn  (1991),  who  conjecture  that  "the  act  of  self- 
explaining  may  make  the  tacit  knowledge.. .more  explicit  and  available  for  use"  (p.  101). 
Students'  open-answer  descriptions  of  the  most  helpful  aspect  of  journal  writing,  as  well  as 
analyses  of  journal  content,  support  this  interpretation.  Relationships  between  memory 
capacity,  speed  of  processing,  learner  ability,  and  cognitive  development  may  also  provide 
insight  into  the  mechanism  by  which  journal  writing  appears  to  scaffold  learning.  Snow  and 
Swanson  (1992)  report  that  higher-ability  learners  have  a  larger  memory  capacity  and  faster 
processing  speed  than  do  lower -ability  learners.  Since  research  shows  that  memory  search 
speed  is  critical  in  associative  learning  and  in  forgetting,  learning  tools,  such  as  journal 
writing,  which  focus  on  reducing  the  demands  on  memory  capacity  and  on  maintaining 
attention,  should  indeed  scaffold  the  learner  in  constructing  meaning. 

Students'  ability  to  conceptualize  questions.  Recently,  David  Merrill  said:  "(SJtudents 
are  not  good  at  conceptualizing  questions.... One  of  my  favorite  strategies  in  class  is  to  say, 
'Ask  me  a  question.'  Once  in  a  while  I  get  asked  a  good  question,  but  most  of  the  time  I 
don't.  Consequently,  I  don't  think  students  know  what  to  ask  anyway"  (Gagne  &  Merrill, 
1991,  p.  38).  If  we  expect  students  to  ask  the  questions  that  experts  would  ask,  the  current 
study  supports  Merrill's  conclusion — the  students  asked  questions  (and  generated  self- 
explanations)  about  few  of  the  elements  that  experts  target  as  important  in  the  stories.  But 
more  importantly  this  study  supports  the  theory  that  learners  construct  intermediate  stages 
of  knowledge  (Bereiter  &  Scardamalia,  1992).  Students'  questions,  statements  of  confusion, 
and  self-explanations  provide  the  data  which  allow  the  teacher  to  adapt  the  instruction  to 
the  needs  of  individual  students  as  well  as  the  group.  Thus,  the  teacher  can  work  within 
each  student's  zone  of  proximal  development  (Vygotsky,  1978),  helping  them  develop 
expertise — ultimately  including  the  ability  to  conceptualize  expert  questions.  (For  an 
extended  discussion  of  journals  as  an  instructional  tool,  see  Cole,  1993.) 

Limitations  of  question-answering  systems.  While  some  people  have  tried  to  develop 
question-answering  computer-assisted-instruction  programs  (e.g.,  Ferguson,  Bareiss, 
Birnbaum,  &  Osgood,  1992;  Graesser,  1992),  these  constrain  the  user  to  particular 
questions,  expressed  in  particular  language.  Such  a  system  (cafeteria -style,  natural- 
language,  or  hypertext)  cannot  answer  a  question  which  has  not  been  anticipated  and 
incorporated  into  the  database.  The  range  of  questions  and  comments  in  just  14  students' 
journals  suggests  that  it  would  be  difficult  if  not  impossible  to  identify  all  the  questions  and 
misconceptions  that  can  emerge  as  readers  try  to  comprehend  a  college-level  short  story. 
Because  literature  is  an  ill-defined  domain  (Spiro  &  Jehng,  1990),  the  potential  number  of 
errors  is  almost  infinite — a  factorial  function  of  the  number  of  words,  phrases,  sentences, 
and  uses  of  irony,  allusions,  symbols,  etc. 
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But  even  if  we  could  develop  a  comprehensive  question-answering  program  for  ill  defined 
domains  such  as  Introduction  to  Literature,  the  question  is  whether  it  is  desirable.  Helping 
students  become  independent  learners  requires  that  we  encourage  them  to  develop  the  skills 
and  confidence  to  identify  and  solve  problems.  The  positive  relationship  between  students' 
ratings  of  journal  helpfulness  and  of  how  much  they  learned  about  literature  lends  support 
to  this  view. 

Individualizing  instruction.  Instructional/learning  strategies  which  encourage  students 
to  express  their  naturally  occurring  questions  and  comments  allow  each  learner  to  build 
naturally  on  whatever  prior  knowledge  he/she  has,  while  providing  an  opportunity  for  the 
teacher  to  monitor  the  student's  comprehension,  identify  misconceptions,  and  adjust 
instruction  accordingly.  Gaining  the  type  of  information  provided  by  journals  in  ill-defined 
domains  seems  to  be  particularly  important  when  instruction  must  provide  ideational 
confrontation  to  facilitate  cognitive  change  (Champagne,  Gunstone,  &  Klopfer,  1985b).  In 
placing  the  learner  at  the  heart  of  the  instructional  process  (Resnick,  1983),  this  study 
demonstrated  how  widely  learners'  questions  and  self-explanations  can  vary.  Moreover,  the 
wide  variation  is  consonant  with  chaos  theory,  which  argues  that  the  sensitive  dependence 
on  initial  conditions— e.g.,  learner  knowledge— can  result  in  large,  unexpected  effects— e.g., 
the  specific  questions  and  comments — (Gleick,  1987). 

Reading  comprehension.  The  current  study  supports  the  hypothesis  that  prior 
knowledge  may  be  the  most  critical  aspect  of  learning  and  problem  solving  (Anderson  et  al., 
1977;  Brown,  Campione,  &  Day,  1981;  Novak,  1985;  Rigney,  1978;  Bereiter  &  Scardamalia, 
1992).  Many  of  the  students'  comprehension  problems  related  to  misconceptions  about  or 
lack  of  world  knowledge.  The  most  striking  examples  of  this  related  to  "Hills  Like  White 
Elephants."  Students  thought  they  knew  what  a  white  elephant  is  (they  applied  a  literal 
meaning)  ard  thus  didn't  think  to  consult  the  dictionary  for  additional  meanings  that  might 
shed  critical  light  on  the  story.  Other  comprehension  problems  related  to  world  knowledge 
about  human  motivation,  causes  of  heart  disease,  European  history,  roles  of  men  and 
women  at  various  points  in  history,  representations  of  Death,  etc. 

Social  or  Individual  Level  of  Knowledge  Building? 

Ng  and  Bereiter  (1991)  conjectured  that  instructional  approaches  which  "promote  a 
'community  of  learners'  (Brown  and  Campione,  1990)"  would  be  the  most  likely  ones  to 
foster  "personal-knowledge  building"  (p.  269).  For  many  people,  including  Ng  and  Bereiter, 
a  community  of  learners  means  having  students  work  in  small  groups.  However,  for  years 
people  have  done  collaborative  learning  but  not  necessarily  with  results. 

The  current  study  suggests  that  a  community  of  learners  might  be  promoted  in  ways 
that  do  not  fit  the  typical  notion  of  collaborative  learning.  On  one  level,  journal  writing 
involves  a  community  of  two;  although  I  generally  respond  as  a  coach,  occasionally  a 
student's  journal  leads  me  to  new  insights.  On  a  second  level,  journal  writing  involves  the 
whole  class  (and  even  former  classes)  since  I  draw  on  journal  questions  and  comments  to 
stimulate  class  discussion.  In  class,  I  model,  coach,  and  scaffold  learning,  and  occasionally 
students  also  lead  me  to  new  insights. 

The  mechanism  by  which  students  develop  their  understanding  and  their  ability  to 
answer  their  own  questions  seems  related  to  Vygotsky's  (1978)  concept  of  mind  as  society. 
First,  students  engage  in  social  dialogue,  as  part  of  a  community  of  learners;  then,  in  journal 
writing,  they  internalize  the  type  of  dialogue  that  they  learned  as  part  of  the  larger 
community,  while  maintaining  access  to  the  social  dialogue.  Journal  writing  seems  to  help 
students  expand  their  own  zones  of  proximal  development,  making  their  tacit 
interpretations  explicit  and  thus  available  for  reflection  and  analysis.  Thus,  journal  writing 
can  help  learners  make  the  transition  from  being  dependent  on  others  to  being  able  to  learn 
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on  their  own.  It  helps  them  move  from  what  Vygotsky  (1978)  called  the  other -directed  to  self- 
directed  stages  of  understanding,  u  capability  that  satisfies  goals  of  many  educators  (e.g., 
Brown,  Campione,  &  Day,  1981;  Duffy  &  Jonassen,  1992;  Frase  &  Schwartz,  1975;  R.  M. 
Gagne,  1980;  Jonassen,  1985). 

Contributing  to  one's  own  understanding.  .Journal  writing  gives  each  student  the 
opportunity  to  ask  questions,  try  to  answer  them,  and  express  their  understanding  of  a 
story.  A  few  students  do  not  have  the  opportunity  to  come  up  with  all  the  questions  and 
answers  and  make  it  impossible  for  other  students  to  contribute  to  their  own 
understanding.  (Even  if  they  have  the  opportunity  in  class  or  in  collaborative  groups,  some 
students  don't  take  advantage  of  it  because  they  lack  confidence  or  fear  peer  ridicule.)  In 
describing  their  Jasper  series,  the  Cognition  and  Technology  Group  at  Vanderbilt  (1992) 
emphasized  the  importance  of  each  student  having  the  opportunity  to  contribute.  Journal 
writing,  even  more  than  collaborative  learning,  guarantees  such  opportunities. 

Recommendations  for  Research 

This  study  was  primarily  a  qualitative  study.  Although  it  suggests  that  journal  writing 
can  be  a  very  flexible  cognitive  tool,  it  raised  many  more  questions  than  it  answered.  A  com- 
prehensive research  agenda  on  journal  writing  should  include  at  leas  the  following  issues. 

Duration  (see  above).  A  longitudinal  study  of  journal  writing  is  warranted,  preferably  a 
full  year  of  journal  writing  in  more  than  one  discipline,  with  follow-up  studies  for  at  least  a 
year. 

Group  size  constrained  the  types  of  statistical  analyses  as  well  as  the  statistical  signifi- 
cance of  results.  Research  with  a  larger  group  is  needed. 

Taxonomy.  Feathers  and  White  (1987)  also  found  that  journal  writing  by  itself  is  an 
insufficient  means  of  identifying  levels  of  cognitive/metacognitive  activities.  Future  efforts 
may  have  to  sacrifice  some  ecological  validity  so  that  research  is  not  constrained  by  the 
cognitive  artifacts  in  journals.  This  might  be  done  by  debriefing  students.  Are  the  questions 
real,  or  do  they  merely  reflect  the  style  of  low-confidence  (or  low-risk-taking)  students  who 
are  anxious  about  expressing  self-explanations?  To  what  extent  did  the  students  generate 
self-explanations?  Did  they  perhaps  apply  a  generalization  about  the  author  from  a  head 
note,  as  some  of  my  students  did?  Did  the  students  discuss  the  story?  If  so,  what  cognitive 
activities  did  they  engage  in  between  reading,  discussing,  and  journal  writing?  Are  they 
reciting  a  question,  self  explanation,  or  summary  provided  by  another  person;  applying  a 
concept  suggested  by  another  person;  or  perhaps  problem-solving  by  building  on  the 
discussion?  However,  trying  to  identify  the  levels  of  cognitive  processing  may  require  the  use 
of  think-aloud  protocols  (Chi  et  al.  1989;  Chi,  de  Leeuw,  Chiu,  &  LaVancher,  1991;  Chi  & 
Van  Lehn,  1991;  Hayes  &  Flower,  1980)  and  shorter  stories. 

Achievement.  Docs  journal  writing  affect  achievement?  Are  there  aptitude-treatment- 
outcomes  interactions?  Of  particular  concern  is  whether  students  may  benefit  in  ways  that 
are  not  directly  related  to  stated  course  goals,  especially  in  long-term  development  of  critical 
reading,  thinking,  and  metacognitive  skills.  One  caution  is  important  here.  If  learning 
involves  construction  of  intermediate  stages  of  knowledge,  direct  comparisons  of  journal 
content  with  end-of-term  content  objectives  (as  suggested  by  one  critic  of  a  manuscript  on 
this  topic)  is  unwarranted. 

Transferability  and  domajn.  Is  journal  writing  transferable  to  other  educational 
settings  and  training  contexts  as  an  instructional/learning  strategy  which  benefits  teacher 
and/or  student  (particularly  to  situations  in  which  learners  have  to  extract  meaning  , 
undergo  cognitive  change,  or  solve  problems)? 
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Research  has  yet  to  determine  whether  a  journal-writing  instructional/  learning  strategy 
is  potentially  more*  beneficial  in  some  domains  than  in  others.  Related  research  suggests 
that,  it  might  be.  Identifying  bugs  in  well-defined  procedural  knowledge  such  as  arithmetic 
seems  easier  than  in  ill-defined  domains  (e.g.,  Spiro  et  al.  1988;  Spiro  &  Jehng,  1990)  such 
as  understanding  philosophy  or  complex  college-level  short  stories.  In  arithmetic,  computer 
programs  such  as  J.  II.  Anderson's  (1990)  LISP  Tutor  can  even  monitor  students'  actions, 
intervene,  provide  gu  dance,  etc.  Learning  and  teaching  an  ill-defined  domain  are  much 
more  complicated.  For  example,  a  complex  college-level  short  story  may  not  have  a  single 
"correct"  meaning;  moreover,  the  student  may  have  inferred  a  "correct"  or  highly  tenable 
meaning  from  wrong  or  incomplete  evidence.  (While  a  student  may  also  get  a  correct  answer 
for  the  wrong  reason  in  arithmetic,  it  is  much  easier  to  identify  the  error.)  Thus  the  teacher 
needs  to  know  not  only  the  student's  conclusion,  but  also  his/her  reasoning.  While  solving 
arithmetic  requires  knowledge  of  a  well-defined  procedure,  understanding  a  complex  story  is 
a  dynamic  process  in  which  the  meaning  unfolds  with  each  word,  each  phrase,  each 
sentence,  ....  Comprehension  is  often  complicated  by  the  fact  that  the  author  of  a  college- 
level  short  story  often  delays  or  even  withholds  information  for  aesthetic  or  affective 
purposes. 

Do  teacher/instructional  methods  (e.g.,  discussion,  lecture,  discovery  learning)  affect 
journal  content,  learning  outcomes,  and  students*  evaluations  of  journal  writing? 

On  a  related  issue,  I  have  suggested  that  adjunct  questions  and  CAI  question  systems 
could  not  adequately  anticipate  or  address  students'  emergent  needs  in  an  ill-defined 
domain  such  as  Introduction  to  Literature.  However,  since  journal  writing  is  time-intensive 
for  both  student  and  teacher,  research  should  address  whether  sets  of  questions  derived 
from  student  journals  could  be  incorporated  into  effective  adjunct  questions  or  CAI  question 
systems. 

Effects  of  resources.  How  does  use  of  resources  (e.g.,  "Catechism  for  Stories"  and  the 
tutorials  Writing  Journal  Questions  and  Writing  Journal  Comments )  or  discussing  stories 
with  others  affect  the  content  of  journals,  learning  outcomes,  and  attitudes?  On  the  final 
survey,  most  students  said  they  would  have  liked  more  direction  in  journal  writing;  yet  only 
one  student  took  advantage  of  the  opportunity  to  use  the  tutorials.  Should  use  of  such 
resources  be  required?. 

Learner  control.  To  what  extent  does  learner  control  influence  achievement,  attitudes, 
outcomes  (e.g.,  independent  learning),  etc.?  Journals  which  encourage  the  learner  to  identify 
and  try  to  solve  his/her  own  problems  in  comprehending  an  assignment  give  learners  more 
control  than  journals  which  require  students  to  respond  to  adjunct  questions  (e.g.,  Hettich, 
1990)  or  engage  in  dialectical  reasoning  about  terms  which  the  teacher  assigns  (e.g.,  Jolley 
&  Mitchell,  1990). 

MotacognttivQ  awareness.  Findings  suggest  that  journal  writing  may  have  helped 
develop  students'  critical  reading/thinking  skills  and  matacognitive  awareness.  For 
example,  one  of  the  students  in  the  class  was  a  36-year-old  woman  who  read  at  the  16.9 
grade  level;  she  had  been  in  my  second-semester  composition  course,  where  her  probing 
intellect  was  evidenced  in  every  essay  she  wrote.  Nevertheless,  in  one  of  her  last  journals  in 
this  study,  she  wrote: 

Yes,  these  journals  are  a  help  to  me..., As  much  as  I  read  I've  never  learned  to  read  criti- 
cally..,.The  point  of  all  this  is  that  I  never  learned  (bothered)  to  take  a  story  apart  and  look 
at  it.  I've  always  accidentally  picked  out  a  phrase  or  image  that  was  especially  appealing, 
but  now  I'm  doing  it  more  often,  and  I'm  spending  more  time  contemplating  the  author's 
motive,  even  in  my  "fun"  reading. 

Although  she  was  by  habit  reflective,  she  had  not  developed  high-level  skills  to  analyze  lit 
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crature.  To  what,  extent  docs  journal  writing  contribute  to  such  changes?  What 
meefianisms  are  involved?  Are  there  aptitude-troatnumt  interactions? 

Effect  on  class  discussion.  Several  students  observed  that  having  to  write  journals 
made  them  think  about  assignments  before  they  came  to  class.  Does  having  to  write  a 
journal  impact  the  quality  and  quantity  of  students'  questions  and  comments  during  class 
discussion? 

Field  articulation.  The  current  study  suggests  that  journal  writing  may  develop  the 
ability  to  attend  to  details  in  learners  who  are  extremely  field  dependent.  Will  empirical 
research  support  this  finding,  and  if  so,  with  what  populations,  in  what  domains,  and  under 
what  conditions?  Would  field-dependent  learners  benefit  more  by  having  to  respond  to 
adjunct  questions  to  help  them  focus  on  salient  elements  (e.g.,  a  set  of  story-specific  study 
questions;  conflicting  accounts  of  a  person,  event,  or  situation;  or  questions  and  comments 
derived  from  other  students' journals)? 

Reading  strategies.  What  strategies  do  high-  and  lower-ability  college  readers  use  in 
Introduction  to  Literature,  when,  and  why?  How  do  these  relate  to  journal  content,  learning 
outcomes,  etc.?  Can  the  strategies  of  high-ability  readers  be  taught  to  lower-ability  readers? 
Why  do  some  students  attempt  to  answer  questions  they  ask  in  journals  while  others  do 
not?  How  do  differences  relate  to  reading  ability,  goal  orientation,  confidence,  willingness  to 
take  risks,  world  knowledge,  etc.9 

Feedback  and  grading.  Do  the  type  and  timing  of  feedback  and  grading  (e.g.,  rigorous 
grading  vs.  no  grading;  immediate  vs.  end  of  term)  affect  the  content  of  journals,  learning 
outcomes,  students'  perceptions  of  journal  helpfulness,  or  their  motivation  to  write  journals? 
(See  Cole,  1992,  for  discussion  of  these  issues.) 

Role  of  world  knowledge.  As  discussed  above,  many  have  hypothesized  that  world 
knowledge  plays  a  dominant  role  in  comprehending  expository  prose.  Findings  in  this  study 
suggest  that  world  knowledge  also  plays  a  dominant  role  in  college  students'  comprehension 
of  complex  short  stories.  How  can  relevant  world  knowledge  be  identified  and  measured? 
What  instructional  strategies  are  most  effective  for  providing  world  knowledge  which  ih 
relevant  to  a  given  story  (e.g.,  isolated  teaching  of  knowledge  before  students  read  a  story, 
footnote  or  marginal  glosses,  hypermedia  programs  to  provide1  access  to  relevant  knowledge 
on  demand,  or  class  discussion  after  students  have  read  a  story)?  Are  there  aptitude- 
treatment  interactions? 

This  paper  has  described  journal  writing  as  a  dynamic  and  flexible  tool  that  is  valued 
by  learners  of  diverse  abilities.  However,  much  research  is  needed  before  we  have?  a  clear 
understanding  of  the  constraints  which  must  inform  instructional  prescriptions  for  journal 
writing. 
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1  Since  I  provide  written  and  oral  instruction  and  practice  in  class,  the  main  purpose  of  the  tutorials  in  my 
classes  is  to  increase  students'  confidence  in  their  ability  to  write  productive  journals  by  providing  opportunity 
for  review,  additional  practice  and  feedback  (particularly  by  allowing  students  to  compare  their  practice  entries  to 
a  wide  range  of  responses  from  former  students).  The  tutorials — Writing  Journal  Questions  and  Writing  Journal 
Comments — identify  the  purposes  and  criteria  of  the  types  of  journals  assigned,  and  provide  practice  identifying 
productive  journal  responses,  and  practice  writing  responses  to  excerpts  from  stories.  They  also  explain  and 
illustrate  many  types  of  journal  entries  (e.g..  state  the  effect  of  the  story  on  the  reader,  paraphrase,  summarize, 
predict,  verify,  relate  story  events  to  personal  experience). 

^Thc  five  stories  were  sequenced:  (1)  "Appointment  in  Samarra."  anonymous;  (2)  "Story  of  an  Hour."  Kale 
Chopin;  (3)  "Hills  Like  White  Elephants."  Ernest  Hemingway;  (4)  "Cathedral,"  Raymond  Carver;  (5)  "The  Cask  of 
Amontillado,"  Edgar  Allan  Poe.  Student  feedback  suggests  that  2  and  4  arc  easiest  to  comprehend  (4  being  easier 
than  2)  and  that  3  is  the  most  difficult. 

**Thc  "Catechism  for  Stories"  also  includes  questions  on  atmosphere,  character,  point  of  view,  crisis, 
climax,  resolution,  tone  and  theme  (and  related  questions  on  keywords,  subject,  plot,  and  symbols).  Following 
arc  the  questions  relating  to  "crisis": 

a.  At  what  point  (scene  or  moment)  can  things  get  better  or  worse  for  the  protagonist?  (Often  this  is  at  the  point 
of  climax.) 

h.  What  was  the  situation  before  this  point? 

c.  What  choices  arc  involved? 

d.  What  arc  the  potential  consequences? 

^Occasionally  I  have  had  students  with  graduate-level  understanding  of  literature;  they  have  been  the  best 
journal  writers.  None  of  the  students  in  this  study  had  such  an  understanding. 
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Note:  An  earlier  version  of  the  study  was  presented  at  the  Annual  Meeting  of  the  American 
Educational  Research  Association  in  Atlanta,  GA,  April,  1993  and  a  full  version  is  currently 
in  press  with  the  Journal  of  Computers  in  Childhood  Education. 

Purpose  of  the  Study 

School  reform  often  looks  to  "technology"  to  help  improve  educat;on  (Texas  Education 
Agency.  1988;  Main  &  Roberts,  1990;  Salehi,  Mullinix,  Wode,  &  Dreighfan,  1980).  Troxel  & 
Grady  (1989)  suggest  that  most  reform  agendas  recommend  the  inclusion  of  technology 
within  the  educational  process.  For  example,  the  Texas  State  Board  of  Edu  cation  (Texas 
Education  Agency,  1988)  developed  a  long-range  plan  for  the  infusion  of  technology  into 
instruction  as  a  means  to  increase  effectiveness  of  the  curriculum.  This  Long-Range  Plan 
gave  schools  with  in  local  districts  the  charge  to  select  and  apply  technology  to  meet  local 
needs.  Other  school  systems  (state  and  local)  have  reported  similar  plans  (Salehi,  et  al, 
1989;  Main  &  Roberts,  1990). 

Burkman  (as  cited  in  Martin  &  Clemente,  1999))  suggested  that  teachers  have 
strategic  roles  in  what  occurs  in  the  classroom.  A  number  of  studies  have  investigated  the 
relationship  of  attitudes  and  implementation  and  use  of  computers  in  educational  settings 
(Savenye,  Davidson,  &  Orr,  1991,  1992;  Sanders  &  Stone,  1986;  Canning,  1989).  In 
addition,  other  reported  results  indicate  that  implementation  and  use  of  computers  is  also 
related  to  student  attitudes  toward  computers  (Campbell  &  Perry,  1989;  Raub,  1981). 
However,  in  today's  schools,  teachers  and  students  are  no  longer  the  only  audiences  that  are 
involved  in  the  decisions  about  implementation  and  use  of  computers.  For  a  variety  of 
reasons,  parents  are  also  becoming  active  participants  in  the  planning  and  implementation 
of  computers  into  the  classroom  (Policy  Research  Project  on  Education,  Technology  and  the 
Texas  Economy,  1989;  Troxel  &  Grady,  1989).  Thus  it  seems  reasonable  to  postulate  that 
perceptions  of  parents  about  technologies  would  also  impact  a  school's  plans  for  technology 
infusion.  Yet  little  research  investigating  attitudes  of  parents  toward  computers  has  been 
reported  in  the  literature. 

The  purpose  of  the  study  was  to  address  the  research  questions:  (1)  How  do  the 
attitudes  of  teachers,  students  and  parents  towards  computers  affect  the  integration  and 
use  of  computer  technology  in  schools?;  (2)  Do  these  attitudes  change  (either  positively  or 
negatively)  or  remain  constant  after  the  implementation  of  technology  into  classroom?;  and 
(3)  What  are  the  implications  of  involving  parents  in  the  planning  of  school  curriculum  and 
activities? 

Background  Information 

Highland  Park  Elementary  School,  located  in  Austin,  Texas,  has  slightly  under  500 
students  and  has  a  relatively  new  building  in  comparison  to  other  schools  in  the  district. 
The  vast  majority  of  the  population  is  white  (an  estimated  5-10%  were  minority)  and 
economically  falls  into  a  high  middle  to  upper  income  area  of  the  city.  Class  sizes  are 
around  21  to  25  and  there  is  low  teacher  turnover.  Administrative  support  of  teachers  is 
high  and,  based  on  informal  observation  and  conversation,  mutual  respect  and  cooperation 
among  teachers  and  administrators  is  high.  According  to  the  Highland  Park  School  report, 
the  school  has  29  local  business  and  community  organizations  as  Corporate  partners 
(Adopt -A-School)  and  a  minimum  of  40  volunteers  in  the  Community/Parent  Volunteers 
organization. 

As  a  means  to  continue  their  tradition  of  innovation,  the  school  purchased  10 
Macintosh  computers  through  funds  raised  by  collaborative  efforts  of  the  school,  parents  and 
local  business  adopters.  The  school  joined  in  a  partnering  consortium  with  Apple  Computer, 
Inc.  to  facilitate  the  purchase  of  additional  computers  (up  to  25  Macintoshes)  and 
collaborate  on  the  applications  of  computers  in  classrooms.  The  school  established  a 
computer  committee  to  develop  plans  for  the  integration  and  use  of  computers  in  the 
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curriculum  and  other  school  activities.  The  computer  committee  was  made  up  of  teacher 
representatives  from  all  grade  levels  and  departments  in  the  school,  plus  parents,  and 
community  leaders.  The  committee  developed  plans  for  computer  training  and  lab  access  for 
teachers  parents,  and  students. 

The  committee  invited  the  principal  investigator  to  the  project  meetings  to  assist 
them  in  the  project.  One  outcome  of  these  planning  meetings  was  to  develop  a  means  to 
document  and  describe  the  experience  level,  attitudes  towards  computers,  and  computer 
use  of  parents,  teachers,  and  students  as  the  computer  integration  was  occurring.  It  was 
determined  that  1991  would  serve  as  the  baseline  year  for  gathering  data  on  experience, 
attitudes,  and  patterns  of  use.  Then  in  subsequent  years,  similar  data  could  be  gathered 
and  compared  to  the  1991  information,  noting  any  significant  changes  in  these  areas.  To 
identify  these  attitudes  and  perceptions,  a  survey  study  was  formulated  and  conducted 
during  the  spring  semester  of  1991  and  1992. 
Method 

Study  Site.  The  study  was  conducted  using  parents,  teachers  and  students  of 
Highland  Park  Klementary  School  in  Austin,  Texas  (a  medium  size  urban  school  district  in 
Texas).  The  school  site  and  the  participants  for  this  study  were  not  selected  at  random. 
The  study  was  conducted  at  the  request  of  the  school's  computer  committee  and  was  based, 
in  part,  upon  the  willingness  of  the  school  personnel,  students,  and  parents  to  participate  in 
the  study.  The  study  involved  approximately  475  students  enrolled  in  kindergarten  through 
fifth  grades,  .'14  teachers,  and  230  parents  during  both  data  collection  periods. 

Procedure.  In  January  and  February,  1991,  a  subgroup  of  the  schooFs  computer 
committee  mot  with  the  principal  researcher  to  develop  the  questionnaires  for  the  three 
populations-students,  teachers,  and  parents.  In  March,  1991,  the  final  draft  of  the  surveys 
were  completed  and  reproduced  for  distribution.  In  late  April,  1991,  the  instruments  were 
distributed  to  nil  three  groups.  Teachers  at  each  grade  level  were  responsible  for 
administering  the  survey  to  the  students  as  well  as  completing  their  own  survey 
individually.  With  the  exception  of  kindergarten,  first  grade  and  special  education  classes, 
students  read  and  responded  to  each  item  independently.  Teachers  in  the  kindergarten,  first 
grade,  and  special  education  classes  read  the  items  and  asked  each  individual  student  to 
respond  as  the  teacher  marked  the  response  sheet  accordingly.  The  parent  questionnaires 
were  sent  home  with  the  children.  Care  was  taken  to  see  that  only  one  survey  was  sent  to 
each  home.  Attached  directions  asked  that  only  one  parent  respond  to  the  questionnaire  and 
return  it .  The  surveys  from  all  three  groups  were  returned  within  two  weeks,  and  then 
coded  for  analysis.  One  year  later,  the  questionnaires,  with  some  modifications  were 
administered  to  the  teachers,  students  and  parents  in  a  similar  manner. 

Instruments.  The  questionnaire  instruments  were  developed  for  each 
subpopulation  of  the  school  (Davidson  &  Ritchie,  1992).  The  questionnaires  were  based,  in 
part,  on  items  from  a  preassessment  questionnaire  by  Smith,  Savenye  and  Davidson  (1988) 
and  an  attitude  instrument  by  Savenye,  Davidson,  and  Orr  (1991,  1992).  These 
instruments  consisted  of  demographic  items,  followed  by  items  on  computer  experience  and 
attitudes  and  anxiety  about  computers.  The  teacher  and  parent  surveys  also  included  items 
on  perceptions  about  the  impact  of  computers  on  teachers1  roles  and  students'  self  esteem 
and  issues  about  computer  training  and  workshops  specific  to  that  particular  school. 
Subjects  responded  to  attitude  anxiety  items  using  a  five-point  Likert-type  scale  of  strongly 
agree  to  strongly  disagree.  The  student  survey  contained  a  total  of  21  items;  the  teacher 
survey  contained  45  items  and  the  parent  survey  contained  43  items.  In  an  effort  to 
encourage  frank  responses  to  items,  all  respondents  could  remain  anonymous.  (Note:  Not 
all  of  the  items  are  discussed  in  this  paper;  for  a  full  discussion,  see  technical  report  by 
Davidson  and  Ritchie  (1992)).  All  subjects  used  a  response  sheet  to  record  their  answers. 
Design  and  Data  Analysis 
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Due  to  the  conditions  of  anonymity,  the  surveys  completed  by  the  same  individuals 
in  1991  and  1992  could  not  be  correlated  for  statistical  analysis.  Therefore,  the  three  sets 
of  samples  (touchers,  parents,  and  students)  were  considered  independent  for  all  analyses  of 
variance.  Differences  in  the  mean  responses  for  each  item  related  to  computer  attitudes  and 
anxieties  between  the  1991  and  1992  survey  groups  were  assessed  with  multiple  t-tests. 
The  results  yielded  statistically  significant  differences  for  some  items  within  each  of  the 
three  participant  groups. 

Reliability  coefficients,  calculated  for  each  group  based  on  the  combined  responses  to 
the  survey  items  from  1991  and  1992,  revealed  a  few  items  of  low  reliability.   As  a  result, 
item  37  from  the  teacher  survey,  items  35  and  37  from  the  parent  survey,  and  item  9  from 
the  student  survey,  were  excluded  from  the  factor  analysis.  The  final  reliability  coefficients 
(('ronbach's  alpha)  were  .79  for  the  student  and  parent  surveys,  and  .87  for  the  teacher 
survey, 

Questions  1-  How  do  attitudes  toward  computers  affect  the  integration  and  use  of 
computers  technology  in  schools? 

Question  2  -  Do  these  attitudes  change  or  remain  constant  after  the 
implementation  of  technology  into  classroom? 

Student  Group.  The  entire  student  body  of  Highland  Park  Elementary  was  almost 
equally  divided  between  Renders  for  both  years  1991  and  1992.  Based  on  the  demographics 
collected  in  the  survey,  kmdergartners  and  first  graders  comprised  the  greatest  portion  of  the 
student  population  (37%)  in  1991  and  (48%)  in  1992,  with  a  relatively  even  split  in  the 
remaining  grades.  Almost  all  students  had  used  a  computer  before  the  survey  was  taken  in 
the  spring,  1991 .  A  little  more  than  half  of  the  student  body  had  a  computer  at  home  (58%) 
in  1991  and  this  percentage  increased  to  68%  by  the  end  of  1992. 

The  strongest  attitudes  about  computers  among  students  wore  positively  expressed 
in  two  items:  Students  (95%  in  1991  and  99%  in  1992)  definitely  believed  that  they  could 
learn  how  to  use  a  computer,  and  they  (96%  in  1991  and  98%  in  1992)  also  expressed  that 
they  would  like  to  use  a  computer  at  home.  These  responses  (items  10  and  15,  respectively) 
are  shown  in  Table  1.  Students  (91%  in  1991  and  95%  in  1992)  also  indicated  that  they 
liked  to  learn  using  a  computer  (item  19)  and  they  (92%  in  1991  and  1992)  were  excited 
about  new  things  in  the  computer  lab  (item  18).  Students  as  a  whole  (93%  in  1991  and  97% 
in  1992)  generally  agreed  that  computer  lessons  are  fun  (item  1 1 ),    Most  students 
responded  that  they  liked  to  experiment  on  the  computers. 

The  1991  data  indicated  that  most  students  were  comfortable  with  computer 
interaction,  but  there  were  still  some  exceptions.  For  example,  kindergarten -first  graders 
(35%)  responded  that  they  were  afraid  to  toueh  a  computer  in  1991.  (However,  only  three 
percent  of  the  1992  respondents  were  afraid  to  touch  a  computer.)  Kindergarten-first  and 
fifth  graders  (30%)  feared  that  they  might  break  or  damage  a  computer,  Some  Kindergarten- 
first  graders  (32%)  also  stated  that  they  got  nervous  when  people  talked  about  computers. 
About  a  third  of  the  student  body  did  not  like  to  rend  about  computers.  Interestingly,  first 
graders  expressed  the  most  positive  attitudes  towards  reading  about  computers  in  1991, 
Students  as  a  whole  (87%)  think  that  computer  lessons  are  interesting  and  84%  of  them 
would  like  to  have  more  computer  lessons  each  week.  However,  21%  of  the  kindergarten-first 
graders  were  not  as  interested  in  computer  lessons  in  1991.  Similarly,  20%  of  the  fourth 
graders  did  not  want  more  computer  lessons  each  week  in  1991,  and  this  percentage 
remained  relatively  stable  (23%)  in  1992  (item  14). 

Most  students  (85%  in  1991  and  1992)  thought  learning  is  more  exciting  with 
computers  (see  item  20),  Although  21%  of  the  second  graders  in  1991  did  not  necessarily 
agree  with  this  response.  Combining  the  responses  of  strongly  agree  and  agree*  a  majority  of 
students  (72%  in  1991  and  67%  in  1992)  stated  that  they  learn  "better"  using  a  computer 
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(item  21).  The  strongest  disagreement  came  from  40%  of  the  fifth  grade  students  and  35%  of 
the  second  grade  students  in  1991.  In  both  years,  more  than  half  of  the  students  responded 
that  they  would  like  to  have  a  job  working  with  computers  someday  (item  17).  Complete 
descriptive  statistics  are  available  from  the  authors  upon  request  (Davidson  &  Ritchie, 
1992). 

Table  1  presents  the  results  of  independent  t-tests  on  each  item  for  the  entire  school 
population  as  a  whole.  Items  5,  6,  and  7  were  intended  to  assess  anxieties  related  to 
computers.  For  each  of  these  items,  the  means  indicate  that  the  student  population 
disagreed  more  strongly  in  1992  than  in  1991.  Stronger  disagreement  with  these  items 
indicates  less  anxiety  about  computer  use.  Items  10,  13,  and  15  were  intended  to  assess 
general  attitudes  towards  computers.  The  means  of  these  items  show  that  the  student 
population  was  in  even  stronger  agreement  in  1992  than  in  1991. 


Table  1 

Independent  T-Tests  for  Student  Group  (K-5):  1991  vs.  1992 


Item 

1991 

1992 

/ 

P 

me  a 

mea 

n 

n 

5.     I  feel  afraid  to  touch  a  computer. 

3.48 

3.75 

2.37 

*<.001 

6.    I  feel  afraid  that  1  might  brenk  or  damage  a  computer. 

3.15 

3.40 

1.31 

*  .003 

7.    I  feel  nervous  when  people  talk  about  computers. 

3.45 

3.71 

1.90 

*<.001 

8.     I  enjoy  reading  about  computers.** 

2.20 

2.16 

1.06 

.516 

9.     I  feel  that  people  who  know  something  about 

2.75 

2.78 

1.01 

.873 

computers  know  more            than  1  do. 

10.  I  can  learn  how  to  use  a  computer.** 

1.38 

1.21 

1.80 

*<.001 

1 1 .  Computer  lessons  are  fun.** 

1.64 

1.56 

1.05 

.601 

12.  I  would  like  to  belong  to  a  computer  club.** 

2.27 

2.17 

1.02 

.867 

13.  I  like  to  experiment  on  the  computer.** 

1.60 

1.44 

1.25 

*  .014 

14.  Schools  should  have  more  computer  lessons  each 

1.72 

1.69 

1.09 

.371 

week.** 

15.  I  would  like  to  use  a  computer  at  home.** 

1.41 

1.29 

1.26 

*  .011 

16.  Computers  are  not  interesting  to  me. 

3.59 

3.65 

1.04 

.640 

17.  Someday,  1  would  like  to  have  a  job  working  with 

2.25 

2.21 

1.17 

.093 

computers.** 

18.  I  get  excited  about  new  things  in  our  computer  lab.** 

1.62 

1.59 

1.04 

.676 

19.  I  like  to  learn  with  a  computer.** 

1.65 

1.55 

1.04 

.681 

20,  It  is  more  exciting  to  learn  with  a  computer.** 

1.69 

1.64 

1.01 

.925 

21.  I  learn  "better"  using  a  computer.** 

2.01 

2.06 

1.04 

.657 

*p<.05         scale:                    1                2  3 

4 

5 

strongly  agree     agree  neutral  disagree  strongly  disagree 


Teacher  Group.  The?  faculty  of  Highland  Park  Elementary  School  was 
predominantly  female  (88%)  with  a  relatively  even  mix  of  age  groups.  The  demographic 
results  showed  that  only  one  fuculty  member  indicated  being  a  beginning  teacher  with  64% 
of  the  faculty  having  had  six  or  more  yeai<*  of  teaching  experience.  Over  half  (52%)  of  the 
teachers  reported  that  they  had  computers  at  home  with  60%  being  Macintoshes.  Of  the 
remaining  faculty  who  did  not  currently  have  a  computer  at  home,  half  of  them  planned  to 
purchase  one  within  the  next  two  years  with  77%  of  them  planning  to  purchase 
Macintoshes.  All  of  the  faculty  member  had  used  computers  previously  (with  97%  in  1991 
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having  used  a  microcomputer).  The  Apple  II  series  of  computers  and  the  Macintosh  were  the 
computers  most  frequently  used  by  the  faculty,  with  a  small  percentage  (6%)  of  faculty 
having  used  IBM,  or  IBM  compatibles  in  1991.  Their  responses  to  these  items  were  similar 
in  1992.  In  1991,  88%  of  the  faculty  indicated  having  taken  a  computer  course  or  workshop 
(88%);  this  had  increased  to  97%  in  1992.  Even  though  teachers  were  experienced  with 
computes,  most  teachers  (91%  in  1991)  rated  their  experience  with  computers  as  only  good 
or  fair  and  9%  rated  it  as  poor.  In  1992,  no  faculty  rated  their  experience  as  poor  and  6%  felt 
very  experienced. 

Faculty  attitudes  towards  the  use  of  computers  in  education  were  generally  very 
positive  in  both  1991  and  remained  consistently  high  in  1992.  Most  faculty  members  (85% 
in  1991  and  94%  in  1992)  valued  teaching  with  technology  (item  21).  The  majority  of  the 
faculty  strongly  agreed  or  agreed  that  the  use  of  technology  will  enhance  teaching  (item  22- 
97%),  motivation  of  students  (item  32-  97%),  and  improve  student  performance  (item  34- 
88%)  i;»  1991  and  their  responses  were  similar  in  1992.  The  faculty  were  unanimous  in 
believing  that  computer  use  is  of  value  to  students  (combining  responses  of  valuable,  very 
valuable,  and  extremely  valuable)  in  both  1991  and  1992.  Computers  were  seen  as  a  tool  to 
further  individualize  instruction  (item  58).  One-third  of  the  faculty  believed  that  the  use  of 
computers  will  actually  decrease  their  planning  time  and  by  1992,  42%  of  them  believed 
that  computers  would  decrease  their  planning  time.  This  faculty  appeared  confident  that 
computers  would  not  take  over  their  jobs  or  diminish  their  roles  as  teachers.  Nearly  all 
teachers  (97%)  expressed  confidence  in  their  ability  to  learn  how  to  use  a  computer. 

A  few  aspects  of  computers  were  of  concern  to  the  faculty  in  1991.  A  small 
percentage  oi  the  faculty  (9%)  were  actually  afraid  to  touch  a  computer  (item  24).  Some 
faculty  (15%)  expressed  concerns  that  they  might  break  or  damage  a  computer  (item  2,r>). 
Ahout  one-third  of  the  teachers  acknowledged  that  they  were  uncomfortable  with  people  who 
are  knowledgeable  of  computers  (item  28-30)  and  12%  were  tense  when  people  talked  about 
computers  (item  26).  These  concerns  diminished  somewhat  in  1992,  but  not  entirely.  The 
area  of  greatest  concern  was  the  impact  of  computers  on  the  role  of  the  teacher.  A  majority 
(59%)  of  the  faculty  believed  their  role  as  teacher  would  be  more  complex  when  using 
computers  (item  37).  A  large  percentage  of  teachers  (41%)  expressed  some  concern  that 
teachers  may  actually  have  to  compete  with  computer  instruction  in  the  classroom.  One- 
fourth  of  the  teachers  believed  that  use  of  the  computer  would  increase  their  planning  time. 
Teachers  who  expressed  these  concerns  may  believe  that  an  increased  complexity  to  teaching 
is  inevitable,  since  the  faculty  was  unanimous  in  believing  that  instruction  by  computer 
technology  is  here  to  stay.  Responses  for  these  items  in  1992  were  similar. 

Table  2  below  summarizes  the  independent  t-tests  for  items  21  through  37  on  the 
teacher  survey.  Even  though  teachers  began  the  study  showing  positive  attitudes,  Table  2 
shows  that  there  were  significant  positive  changes  in  items,  24,  32,  33,  and  36,  which  relate 
to  attitudes  and  anxieties  toward  computers. 
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Table  2 

Independent  T-tests  for  Teacher  Group:  1991  vs.  1992 


Item 

1991 

1992 

/ 

P 

mea 

mea 

n 

n 

21.       I  like  to  teach  with  computer  technology. 

1.65 

1.73 

1.63 

.168 

22.  I  think  quality  instruction  using  technology  will  only 

1.44 

1.61 

1.02 

.96 

enhance  my  teaching. 

23.  I  value  teaching  with  technology. 

1.44 

1.45 

1.23 

.559 

24.  I  feel  afraid  to  touch  a  computer. 

4.50 

4.64 

2.21 

*.027 

25.  I  feel  afraid  that  1  might  break  or  damage  a  computer. 

4.03 

4.30 

1.44 

.309 

26.  I  feel  tense  when  people  talk  about  computers. 

3.91 

4.06 

1.14 

.715 

27.  I  enjoy  reading  about  computers. 

3.03 

2.82 

1.10 

.795 

28.  I  feel  intimidated  by  people  who  know  something 

3.47 

3.91 

1.61 

.179 

about  computers. 

29.  I  think  computers  are  dehumanizing. 

4.32 

4.39 

1.26 

.514 

30.  I  fear  that  computers  may  take  over  some  parts  of  a 

4.26 

4.30 

1.49 

.257 

job  that  1  enjoy. 

31.  I  feel  confident  that  I  can  learn  how  to  use  a  computer. 

1.38 

1.42 

1.65 

.157 

32.  I  think  students  are  more  motivated  when  they  can  ' 

1.56 

1.76 

2.39 

*.015 

learn  using      computer  technology. 

33.  I  think  instruction  by  computer  technology  is  just 

4.68 

4.48 

2.25 

*.023 

another  fad. 

34.  I  think  that  using  instruction  via  computer  technology 

1.74 

1.82 

1.26 

.512 

will  help         improve  students'  performance. 

35.  I  think  teachers  compete  with  slick  packages  and  high 

3.85 

3.76 

1.42 

.325 

tech  machines. 

36.  When  utilizing  computers,  the  teacher  becomes 

1.97 

1.58 

2.41 

*.015 

guide/facilitator. 

37.  When  utilizing  computers,  the  teacher's  role  becomes 

2.50 

2.33 

1.34 

.410 

more  complex. 

38.  When  utilizing  computers,  the  teacher  is  able  to  further 

1.94 

1.85 

1.05 

.896 

individualize  instruction. 

39.  When  utilizing  computers,  the  teacher's  role  is 

4.44 

4.48 

1.02 

.960 

diminished. 

*p<05          scale:                    1               2  3 

4 

5 

strongly  agree       agree    neutral    disagree  strongly 

disagree 


Parent  Group. ..Parents  responding  to  the  questionnaire  were  predominantly 
female  (69%  in  1991  and  73%  in  1992)  and  between  the  ages  of  30  and  49  (94%).  The 
majority  of  respondents  (85%)  had  earned  a  bachelor's  degree  or  higher  with  16%  having 
earned  a  Ph.D.  Approximately  61%  had  computers  at  home  in  1991,  and  this  increased  to 
67%  in  1992.  Almost  all  respondents  (94%)  had  used  microcomputers,  with  46%  being  IBM 
personal  computers  in  1991,  a  figure  which  increased  to  58%  in  1992.  Macintosh,  the 
predominant  platform  at  school,  was  second  in  use  and  the  Apple  fl  series  computer  as 
being  the  least  frequently  used.  Nearly  half  of  the  parents  (46%)  considered  themselves 
"good"  or  "very  experienced"  with  computer  technologies  in  1991,  and  this  figure  increased 
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slightly  in  1992.  Parents'  attitudes  toward  computers  were  generally  very  positive.  For 
instance,  when  asked  to  respond  to  the  item  "I  feel  afraid  to  touch  a  computer,"  87%  of  the 
parents  disagreed  or  strongly  disagreed  with  the  statement.  With  the  statement  "  I  feel 
confident  that  I  can  learn  how  to  use  a  computer",  94%  were  in  agreement.  There  remained 
a  small  number  of  respondents  (7-8%)  who  expressed  uncertainties  or  fears  associated  with 
computers.  (See  items  24,  25,  28  &  28.)  The  1992  results  indicated  little  change  in  these 
percentages. 

The  majority  of  parent  respondents  (74%)  indicated  that  they  thought  that  the 
teacher  would  become  more  of  a  guide/facilitator  and  they  also  agreed  that  a  teacher  would 
be  able  to  further  individualize  instruction  using  the  computer.  The  role  of  the  teacher  when 
using  computers  was  viewed  as  more  complex  (47%)  and  not  diminished  by  using  computers 
(81%)  by  a  majority  of  these  respondents.  And  they  also  disagreed  (24%  strongly  disagree 
and  40%  disagree)  with  the  statement  that  "teachers  compete  with  slirk  software  packages 
and  high  tech  machines."  Again,  the  1992  results  indicated  little  change  in  these  attitudes. 

Parental  responses  also  reflected  a  general  belief  that  students  are  more  motivated 
and  would  have  improved  performance  with  computers.  Of  the  total,  79%  of  the  respondents 
either  agreed  or  strongly  agreed  with  the  statement  that  studc>nts  are  more  motivated  when 
they  can  learn  using  computer  technology  (item  32).  With  item  34  which  was  related  to 
improved  students'  performance  using  instruction  via  computer  technology,  70%  were  in 
agreement.  The  parents  placed  strong  value  on  learning  to  use  computers  as  a  part  of  their 
children's  education  (with  97%  either  indicating  valuable,  very  valuable,  or  extremely 
valuable). 

Tables  3  summarizes  the  results  on  independent  t- tests  on  items  24  through  39. 
Only  three  items  (items  30,  32,  and  36)  were  found  to  be  statistically  significant. 
Interestingly,  item  32,  'attitudes  towards  instruction  with  computers',  helps  explain  over 
16%  of  the  variance  between  the  parent  responses  in  1991  and  1992. 
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Table  3 

Independent  T-tests  for  Parent  Group:  1991  vs.  1992 


Item 

1991 

1992 

/ 

p 

me  a 

mea 

n 

n 
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1.47 

1.23 

.109 
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9  14 

1  QQ 

1  *uu 

1  90 

1  79 
.1  16 

will  help          improve*  student*/  performance. 

35.  1  think  teachers  compete  with  slick  packages  and  high 

3.76 

3.81 

1.02 

.880 

tech  machines. 

36.  When  utilizing  computers,  the  teacher  becomes 

2.24 

2.10 

1.37 

*.018 

guide/facilitator. 

37.  When  utilizing  computers,  the  teacher's  role  becomes 

2.68 

2.71 

1.20 

.170 

more  complex. 

38.  When  utilizing  computers,  the  teacher  is  able  to  further 

2.20 

2.10 

1.04 

.745 

individualize  instruction. 

39.  When  utilizing  computers,  the  teacher's  role  is 

4.06 

4.16 

1.27 

.066 

diminished. 

*p<.05          scale:                    1               2  3 

4 

5 

strongly  agree  agree  neutral  disagree  strongly  disagree 
Question  3:  What  are  the  implications  of  involving  parents  in  the  planning  of 
school  curriculum  and  activities? 


Teacher  Group.  Not  surprisingly,  the  computer  experience  of  teachers  increased 
substantially  over  the  one- year  period.  The  faculty  were  unanimous  in  believing  that  the 
computer  was  of  value  to  students  and  there  were  no  significant  differences  among  the  1991 
and  1992  comparisons.  Most  faculty  valued  teaching  with  computers,  and,  by  1992,  42%  of 
them  believed  that  computers  would  decrease  their  planning  time.  Interestingly,  software 
ratings  decreased  dramatically  by  the  faculty,  which  is  similar  to  the  findings  reported  by 
Savenye,  Davidson,  and  Orr  (1992).   However,  teachers'  attitudes  toward  computers  were 
in  general  very  positive  in  1991  and  remained  positive  at  the  end  of  the  study.  Conversely, 
teachers'  anxieties  toward  using  computers  were  relatively  minor  and  remained  unchanged 
at  the  culmination  of  the  study.  The  lack  of  change  may  be  due,  in  part,  to  a  ceiling  effect. 
Teacher  attitudes  were  very  high  and  anxieties  Were  very  low  in  1991.  Except  for  a  few 
individual  items,  no  significant  changes  occurred  in  1992.  The  changes  that  were  noted 
indicated  even  more  positive  attitudes  and  lower  anxieties. 
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Student  Group.  The  majority  of  the  students  reported  some  computer  experience 
with  more  than  half  reported  having  a  computer  at  home.  Attitudes  about  computers 
among  the  total  student  population  of  the  school  were  generally  positive.  For  the  most  part, 
students  showed  low  anxiety  toward  computers  in  1991.  Even  so,  students  showed  a 
significant  decrease  in  their  anxiety  towaH  computers  at  the  culmination  of  the  study  in 
1992.  Accompanying  this  change  was  a  significant  increase  in  students'  general  attitudes 
towards  computers.  As  students'  computer  anxiety  lessened,  their  interest  in  using  and 
learning  with  computer  apparently  increased, 

Parent  Group*  Two  thirds  of  the  parents  had  computers  in  the  home  and  most 
consider  theifiselves  experienced  with  computers.  Attitudes  towards  computers  were 
generally  very  positive  and  changed  little  between  11)91  and  1992.  A  majority  of  the 
parents  believed  that  the  emphasis  on  computer  use  at  the  school  was  at  the?  appropriate 
level  and  also  believed  that  computes  have  a  positive  impiiet  on  children's  self  esteem. 
Summary  of  Results 

Each  of  the  three  groups  reported  successful  experiences  with  the  computer  and  that 
their  attitudes  were  positive  about  computers  in  both  1991  and  1992.  Whether  it  wan 
because  attitudes  were  high  that  the  changes  in  experience  increased  with  each  group  or 
because  the  groups  tended  to  be  experienced  with  computers  that  attitudes  remained 
positive  or  increased  remains  the  question.  With  this  study,  we  seemed  to  have  run  into  a 
chicken-and-egg  type  of  dilemma,  that  is,  which  came  first  (attitudes  or  successful 
experience)  and  had  influence  over  the  other.  We  speculate  that  it  may  have  been  a  little  hit 
of  both.  Soliciting  and  sustaining  the  involvement  of  students,  teachers,  and  parents  in  the 
implementation  of  computer  technologies  may  have  resulted  in  a  synergistic  effect  which 
augmented  the  already  positive  attitudes  of  the  Highland  Park  Elementary  School 
community  towards  technology.  Therefore,  one  strong  implication  of  this  study  may  be  that 
serious  consideration  of  the  notion  of  school  community  involvement  must  be  given  when 
planning  for  the  integration  of  computers  (or  any  innovation)  in  schools. 

When  a  total  school  community  is  being  surveyed  about  attitudes  and  perceptions 
toward  computers,  the  complexity  of  variables  is  increased  and  thus  leads  to  a  richness  in 
information  about  integrating  computers  into  a  school  environment.  By  considering  parental 
attitudes  in  addition  to  teacher  and  student,  perhaps  a  better  understanding  of  community 
support  will  be  established.  This  understanding  could  help  build  widespread  support  for 
implementation  and  infusion  of  computer  technologies  within  the  school. 

However,  the  success  at  Highland  Park  Elementary  indicated  by  this  study  is  also 
due,  in  part,  to  measures  taken  by  school  leaders  during  the  planning  and  implementation 
of  computer  technology.  Teachers  were  allowed,  and  encouraged,  to  take  school  computers 
home  for  self-paced  training  during  the  summer  of  1991 .  Over  60%  of  the  teachers  availed 
themselves  to  this  opportunity.  Strong  collegial  relationships  helped  support  general  on- 
going in-house  training;  teachers  continually  trained  nther  teachers  informally.  Faculty  were 
offered  over  20  inservice  computer  training  opportunities  conducted  by  teacher  leaders, 
corporate,  and  university  personnel.  In  addition,  parents  were  offered  the  opportunity  to 
participate  in  school-sponsored  computer  training  sessions.  Parents  were  very  involved  in 
raising  moneys  for  the  computer  and  actively  volunteered  for  after  school  computer  clubs 
with  students.  Thus,  results  tend  to  confirm  the  belief  that  integration  of  technology 
requires  a  supportive  environment  from  administrators,  fellow  colleagues,  and  the 
community. 

While  the  resulls  of  the  study  are  extremely  encouraging,  one  must  be  cautious 
about  drawing  generalizations  and  inferences  about  the  reported  findings  to  other  school 
populations  or  environments.  This  caution  is  due  in  part  to  the  nature  of  the  design  of  the 
study  and  the  nonrandomization  of  the  participant  selection.  For  the  most  part,  the  three 
groups  surveyed  may  not  be  typical  or  representative  of  the  population  at  large.  For 
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instance,  the  faculty  were  very  experienced  in  teaching,  a  large  percent  of  the  children  (at  all 
grade  levels)  had  experience  with  computers  with  half  having  a  computer  in  the  home,  and  a 
significant  number  of  the  parents  had  college  degrees  or  higher. 

The  outcomes  of  the  survey  show  promise  that  computers  can  be  effectively 
implemented  into  the  classroom  with  student,  teacher,  and  parent  attitudes  remaining 
positive  and  their  experiences,  successful.  It  is  recommended  that  additional  longitudinal 
studies  be  designed  and  conducted  to  further  substantiate  these  findings. 
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Introduction 

While  most  graduate  programs  in  educational  technology  focus  on  the  design  of  instructional 
materials  in  a  wide  range  of  media,  the  most  common  delivery  mode  of  education  and 
training  is  still  the  instructor-led  course.  Regardless  of  how  much  CBT  or  multimedia 
experience  a  designer  may  have,  he  or  she  is  often  called  upon  to  put  together  a  workshop  or 
presentation  quickly.  To  enable  teachers  and  trainers  to  draft  and  edit  lesson  plans 
efficiently  and  to  share  them  with  others  in  a  consistent  forma    a  tool  called  PLANalyst  was 
developed. 

This  paper  describes  the  process  used  to  design  and  evaluate  a  computer-based  tool  that 
guides  the  development  of  lesson  plans.  The  tool,  PLANalyst,  contains  a  small  expert  system 
with  a  human  face  which  analyzes  lessons  and  gives  specific  feedback  for  improvement.  The 
process  used  to  create  PLANalyst  can  be  generalized  to  the  development  of  tools  to  support 
any  complex  intellectual  task. 

PLANalyst  runs  on  Macintosh  computers  and  was  constructed  with  HyperCard.  No 
knowledge  of  HyperCard  is  required  other  than  familiarity  with  the  graphical  user  interface 
conventions  of  mouse  and  menu  use. 

The  Problem 

What  ..lakes  planning  a  class  or  workshop  or  presentation  a  challenge?  What  gets  in  the  way 
of  good  instruction?  Novice  instructors  tend  to  try  to  pack  too  much  information  into  the 
time  allotted,  don't  provide  enough  opportunities  for  distributed  practice  and  feedback,  forget 
that  many  learners  need  to  be  systematically  motivated  by  having  the  relevance  of  the 
content  made  clear,  neglect  to  orient  the  learners  to  the  material  and  activate  their  prior 
knowledge,  and  stick  to  a  limited  number  of  tried  and  true  teaching  strategies.  Seasoned 
instructors  do  all  of  these  things,  too,  though  less  often. 

These  mistakes  can  be  avoided  by  devoting  enough  effort  to  the  planning  of  a  lesson.  As  with 
anything  else,  this  kind  of  planning  effort  will  happen  more  in  the  real  world  if  the  task  can 
be  made  easy,  efficient,  and  (dare  we  hope)  enjoyable. 

PLANalyst  was  developed  to  support  the  lesson  design  process,  It... 


allows  easy  creation  and  editing  of  lessons:  changing  times,  adding  and  deleting 
activities,  rearranging  the  sequence,  etc. 

provides  clearly  printed  out  lesson  plans  which  show  minute  by  minute  how  the 
lesson  is  to  be  presented. 

encourages  users  to  base  their  lessons  on  sound  instructional  principles  and  to 
realistically  estimate  their  time  requirements. 

contains  a  built-in  expert  system  that  can  evaluate  lesson  plan  and  suggest 
changes. 

provides  a  common  language  and  tool  for  team  teaching  and  instructional 
development.  The  work  of  designing  a  lesson  can  be  spread  out  over  multiple 
authors.  PLANalyst  allows  event?-,  from  separate  lesson  plans  to  be  merged  into 
one. 
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can  In*  u.scil  Id  promote  consistency  across  classes  with  multiple  sections  and 
instructors. 

can  he  used  to  i iIihc*i-vi*.  record,  document  and  evaluate  teaching  by  principals, 
teacher  educators  and  training  managers. 

Steps  in  Creating  an  Electronic  Performance  Support  Tool. 

The  process  of  designing  a  tool  like  PlJVNalyst  follows  a  path  similar  to  that  performance 
technologists  use  to  develop  other  problem  solutions. 

1.  Define  the  problem. 

2.  Develop  solution  specifications. 

3.  Develop  the  interface 

4.  Develop  the  knowledge  base. 

Across  all  of  these  steps  is  a  constant  process  of  user  testing  and  refinement  as  needed. 

The  problem  definition  in  this  case  included  the  assumptions  that  the  target  users  of  the 
tool... 

are  sometimes  unsure  of  the  necessary  ingredients  of  a  successful  lesson 

often  forget  tc  include  those  elements  they  know  to  constitute  good  instruction 

often  err  in  the  amount  of  time  allocated  to  instructional  activities 

need  to  be  able  to  communicate  their  instructional  intentions  clearly  to  others  on 

paper 

see  lesson  planning  as  an  onerous  chore  to  be  done  with  minimal  detail  and  all  in 
one  draft 

would  benefit  by  exposure  to  teaching  strategies  they  might  not  normally  use 

The  solution  specification  included  the  following  characteristics  of  the  tool.  The  user  should 
be  able  to... 

generate  lesson  ideas  quickly  and  refine  them  later 
rearrange  the  order  of  lesson  activities  easily 

consider  time  requirements  at  every  point  while  drafting  the  lesson. 

create  attractive  printouts  in  various  formfi 

justify  the  instructional  purpose  served  by  every  instructor  act. 

access  specific  suggestions  for  teaching  strategies  with  pointers  to  published 

articles  and  other  documents 

Design 

PLANalyst  is  built  on  the  assumption  that  a  presentation  or  lesson  can  be  divided  into 
distinct  chunks  of  one  minute  or  more  in  duration,  and  that  each  chunk  should  serve  some 
instructional  or  practical  purpose.  PLANalyst  is  intended  to  be  equally  useful  to  experienced 
and  novice  presenters  or  teachers,  though  meeting  this  design  goal  has  proven  to  be  a 
challenge. 
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The  instructional  design  model  that  PLANalyst  is  based  on  most  closely  is  *hat  of  Gagne's 
nine  events  of  instruction  (Gagne,  Briggs  &  Wager,  1992).  That  model  is  most  appropriately 
used  for  direct  instruction  ...the  teacher-centered,  objective-driven  approach  used  most  often 
in  the  training  of  adults.  Additionally,  suggestions  summarized  from  the  adult  training 
literature  and  professional  practice  by  Powers  (1992)  were  incorporated.  To  widen  the 
applicability  of  the  tool  to  teachers  of  children,  models  of  indirect  instruction  and  cooperative1 
learning  were  added  to  the  rule  base.  These  models  were  derived  from  work  by  Johnson  & 
Johnson  (1991),  Joyce,  Weil  &  Showers  (1992),  and  Slavin  (1983). 

In  one  fundamental  way,  however,  PLANalyst  runs  contrary  to  the  standard  instructional 
systems  design  approach:  it  allows  one  to  begin  the  process  of  lesson  design  by  listing 
possible  activities  rather  than  proceeding  deductively  from  objectives.  This  activities-first 
approach  is  in  keeping  with  how  teachers  conceptualize  the  lesson  planning  process 
(Sherman,  Driscoll  &  Klein,  1994),  There  is  nothing  to  prevent  an  instructional  designer 
from  using  PLANalyst  by  specifying  objectives  first  and  going  on  to  draft  learning  activities. 

There  are  three  major  parts  to  the  PLANalyst  tool: 

preparation  screens, 
event  screens,  and 
an  expert  system. 


On  Preparation  Screens,  the  lesson  designer  describes  the  learners,  the  content,  the 
instructional  context,  and  practical  considerations  such  as  room  arrangements  and  materials 
needed.  This  description  can  he  printed  out  when  the  plan  is  finalized  to  serve  as  a  reminder 
before  the  lesson  is  delivered.  There  are  three  such  screens  in  the  present  version  of 
PLANalyst.  The  presence  of  fields  and  checkboxes  requiring  input  reminds  the  user  to 
analyze  the  learners  and  subject  matter  before  rushing  headlong  into  design. 
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PLAN  -  EOTEC  470  Week  I 

 S 


PLANalyst 


A  Le«on  Design  Tool 


Lesson  Name 


470  Week  t 


Bernie  Dodge  et  ol 


Jeroacha  Elementary,  lab  vith  IS  LC  M  s.  Student 
teachers  in  second  semester  of  block. 


2:15  PM    to    4:42  PM  1 


ill 

ni 

D 

Eueiits 

Getting  Started 
Fill  out  survey 
Read  syllabus 
About  470  (Slide  1) 
Importenoe  (Slide  2) 
Information  Aoje  (Slides  3-4) 
Ed  Tech  (Slides  5-6) 
Objectives  (Slides  7-10) 
Describe  Readings 
Describe  Assignments 
Q  &  A  about  course 
Break 

Intro  to  Macintosh 
Macintosh  Tour 
Debrief  Mac  Tour 
Introduce  HS  2000 
Shov  High  School  2000 
Debrief  HS  200CL 
For  Next  Veek^J*7 


To  describe  each  event,  the  user  completes  an  Event  Screen  like  the  one  below. 


PLAN  -  EDTEC  470  Week  1 


Introduce  HS2000 


4:11    to  4:21 


Motiuotion 
•  Arouse  Uncertainty 


Present  something  that  seems  to  contradict  what  the 
learners  already  know.  (Be  sure  to  allow  the 
learners  to  resolve  the  uncertainty  *^onr*  point.) 


Uuiatiun  ■  Desmption 


Ask  then  to  imagine  this  year's  second  graders  as  seniors  in  the  year  2003 
With  the  technology  that  vill  be  available  then,  how  would  you  teach 
Shakespeare »  Vhat  will  the  computers  look  fee?  What  will  the  classroom 
look  like? 

Ask  them  to  think  irdividually .  then  brainstorm  in  pars ,  then  share  list 


link  In 

sid<k... 


ODD 


Kach  event  is  labeled  with  a  short  name  (Introduce  HS  2000,  in  this  example).  A 
h  erarchical  pop  up  menu  allows  the  user  to  tan  the  event  with  one  of  close  to  70  identifiers 
which  describe  the  instructional  purpose  of  the  event  (in  this  example  "Arouse  Uncertainty", 
or  e  of  10  items  under  the  "Motivation"  category  was  chosen. 
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3d 


PLAN    EDTEC  470  Week  1 


in 


Introduce  HS  2000 


4:11   to  4:21 


Mot 
•  f 


Mn  I  MM  I  Mill 


Qbjecttues 

Pi eieijuiiile  Knowledge 
InrmmAlion  Presentation 
Pint  lice  0  feedback 
lesttnn 

Enrichment  U*  Remediation 
lilhar 


Mrdiijin 


1MV  ■!*♦■' 


Aik  limn  to  lh«*  individu.lly ,  th» 


Noueity 

► 

► 
► 


eleuence  | 


Inuoke  Mystery 
Cite  Model 

Establish  Future  Releuence 
Establish  Past  Releuance 
Raise  Confidence 
Establish  Reward 
Establish  Credibility 
Establish  Need 


♦ 

link  In 

SI.M 

The  user  specifics  how  lung  I  hi'  event  will  take  by  clicking  on  arrow  buttons  and  the  time 
span  during  the  which  the  event  will  take  place  is  immediately  recalculated  and  displayed  at 
the  upper  right  part  of  the  screen.  The  largest  item  on  the  screen  is  a  scrolling  field  in  which 
notes  on  what  to  do  at  this  point  in  the  lesson  can  be  written.  Users  can  navigate  through  the 
lesson  by  clicking  on  the  arrow  buttons  in  the  lower  right  part  of  the  screen,  or  by  clicking  on 
the  "Return  to  Main  Menu"  button. 

On  the  Main  Menu,  the  list  of  all  events  in  the  lesson  can  be  seen.  One  can  travel  to  the 
screen  describing  the  event  by  clicking  on  its  name.  Events  can  be  re-sequenced  by  dragging 
them  up  or  down  through  the  list. 

With  the  Expert  System,  the  user  can  at  any  point  request  advice  from  a  modest  knowledge 
base  built  into  PLANalyst.  The  system  scans  through  the  draft  lesson  plan  and  looks  at  the 
amounts  of  time  spent  on  specific  types  of  events,  the  sequencing  of  those  events,  and 
descriptions  of  the  learner  gr  jp  and  content.  A  rule  base  of  twenty  principles  is  consulted 
which  triggers  positive  comments  or  corrective  feedback  as  appropriate.  The  advice  is  given 
through  the  persona  of  Anna  Liszt,  who  is  described  as  an  experienced  teacher  and 
instructional  designer.  This  feedback  can  be  viewed  on  screen  or  printed  out. 
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Vou'ue  prouided  some  prat  1KB  uHthm 
the  lesson,  but  I  wonder  if  youue 


prouided  it  soon  enough.  It  tf»ernt  u% 
though  there  Is  a  long  stiofrh  ol 


information  being  pretentud  uuloie 
your  learners  haue  a  rhnnc e  in  le%t 


their  understanding  ol  It. 


Perhaps  you  can  break  tip  tin* 
information  into  subpart!  and  gme 
learners  practice  at  one  piece  at  a 
time? 


♦ 
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Field  Testing 

PLANalyst  has  bevn  used  by  close  to  two  hundred  students  ut  Sim  Diego  State  University 
during  the  hist  two  years.  About  "half  of  those  using  the1  tool  are  preservice  teachers  taking  a 
course  in  educational  computing;  the  others  are  graduate  students  in  instructional  design. 
At  the  end  of  each  semester,  students  rate  each  element  of  PLANalyst  and  suggest 
improvements. 

Routines  in  the  software  (Williams  &  Dodge,  1993)  were  used  to  unobtrusively  track  data  on 
how  PLANalyst  was  used  by  students.  These  data  include  the  amount  of  time  spent  in  each 
part  of  the  program,  the  amount  and  type  of  editing  done,  and  patterns  of  use  of  the  expert 
system. 

A  typical  sample  of  the  tracking  data  looks  like  this: 

2841218642  SO  with  PLANalyst  version  .99c3 

2841218648  MBR 

2841218813  AS  8:00  AM 

2841218819  AE 

2841218831  ED  Take  Roll  to  3 

2841218878  EW  Take  Roll  .#  Words  =  9 

2841218892  AE 

284121891 1  ED  Importance  of  Chinese  Society  to  8 

Which  translates  into  tv 

Wednesday,  January  12,  1994  11:24:02  AM 

1 1 :24    Opened  stack  with  PLANalyst  version  .99c3 

11:24    Menu  choice:  Brainstorin  Events 

1 1 :26    Adjusted  starting  time  to:  8:00  AM 


179 


11:26  Clicked  on  Events  list 

11:27  Adjusted  duration  of  Event  Take  Roll  to  3  minutes 

1 1:27  Edited  Description  of  Event  Take  Roll  .  Total  #  Words  =  9 

1 1:28  Clicked  on  Events  list 

1 1:28  Adjust  duration  of  Event  Importance  of  Chinese  Society  to  8  minutes 

In  the  next  round  of  examination  of  these  data  now  under  way,  cluster  analysis  will  be  used 
to  identify  categories  of  usage  patterns,  and  the  completed  lesson  plans  of  all  participants 
will  be  rated  qualitatively  and  related  to  these  clusters. 

In  the  earliest  trials  of  the  software,  user  feedback  was  collected  in  the  form  of  open-ended 
paragraphs  summarizing  their  opinions,  as  well  as  informal  interviews  with  users.  In  the 
most  recent  field  tests  of  PLANalyst,  a  questionnaire  was  used  to  get  specific  feedback  on  the 
program  (Appendix  B).  The  results  of  testing  the  most  recent  version  were  as  follows: 
(N=l  1).  To  simplify  the  data  for  a  coarse-grained  analysis,  Agree  and  Strongly  Agree 
responses  were  lumped  together,  t<s  were  Disagree  and  Strongly  Disagree;  Neutral  responses 
were  left  out  of  the  table. 


Statement 

% 

% 

Agre 
e 

Disagree 

The  printouts  were  legible  and  attractive 

91% 

0% 

The  Rearrange  Events  feature  is  useful 

91% 

9% 

The  Brainstorming  Events  feature  is  useful 

91% 

9% 

PLANalyst  was  easy  to  use 

91% 

0% 

PLANalyst  will  be  useful  primarily  for  planning  special  or  unusual 

90% 

0% 

events 

The  time  spent  putting  a  lesson  into  PLANalyst  form  would  be 

54% 

36% 

well  spent 

The  program  seemed  to  run  slowly 

50% 

2  7% 

It  was  difficult  to  find  an  appropriate  category  to  describe  an 

45% 

36% 

event 

I  wanted  to  include 

There  were  enough  different  print  out  formats  provided 

45% 

9% 

The  advice  from  Anna  was  useful  and  appropriate 

36% 

45% 

I'd  xise  "Lite"  mode  more  often  than  the  Full  mode 

36% 

0% 

PLANalyst  is  too  time-consuming  to  be  practical 

36% 

45% 

In  general,  a  lesson  should  fit  on  one  page 

36% 

36% 

PLANalyst  will  be  useful  for  everyday  planning 

36% 

36% 

The  program  was  cumbersome 

20% 

70% 

The  type  on  the  printouts  was  too  small 

9% 

81% 

The  printouts  took  up  too  much  paper 

9% 

63% 

The  results  in  the  table  above  break  down  cleanly  into  three  categories:  at  the  top,  desirable 
statements  that  most  users  agreed  with;  at  the  bottom,  undesirable  statements  that  users 
tended  to  disagree  with,  and  the  middle  ground  where  targets  of  improvement  can  be 
identified.  The  statements  shown  in  bold  seem  the  most  important  to  work  on,  and  the  next 
round  of  software  refinement  will  focus  on  raising  the  ratings  here.  In  practice,  this  means 
adding  more  categories  for  describing  instructional  events  and  adding  rules  to  Anna's 
rulebase  to  smarten  her  advice. 
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One  major  step  in  this  regard  is  the  addition  of  a  database  of  teaching  strategies.  Over  600 
ERIC  citations  have  been  identified  as  containing  information  on  useful  teaching  techniques. 
In  the  final  release  version  of  PLANalyst,  users  will  be  able  to  access  this  database  and  print 
out  strategies  and  citations  appropriate  to  their  present  lesson.  It  is  hoped  that  the  database 
will  be  rich  enough  that  even  experienced  teachers  and  trainers  will  be  shown  strategies  that 
they  would  not  have  thought  of,  thus  raising  the  perceived  usefulness  of  Anna's  advice  and 
the  global  value  of  the  tool. 
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Appendix  A:  Instructional  Event  Categories 

Prior  Knowledge  Activation 

Review  Previous  Learning 

Use  Analogy 

Ask  Review  Questions 

PreTest 

Information  Preview 

State  Objectives  Formally 
State  Objectives  Informally 
Show  Final  Goal 
Provide  Overview 
Provide  Background 
Teacher  Pre-Questions 
Learner  Pre-Questions 
Advance  Organizer 

Motivation 

Novelty 

Arouse  Uncertainty 
Evoke  Mystery 
Cite  Model 

Establish  Future  Relevance 
Establish  Relevance  to  Past 
Raise  Confidence 
Establish  Reward 
Establish  Credibility 
Establish  Need 

Information  Acquisition 

Give  Definition 

Examples/Non-examples 

Definitions  &  Examples 

Procedure  Description 

Procedure  Demonstration 

Fact  Presentation 

Problem  Definition 

Discussion 

Socratic  Dialog 

Analogies 

Anecdotes 

Inquiry  -  Individual 

Inquiry  -  Group 

Peer  Tutoring 

Teacher  Mnemonic 

Learner  Mnemonics 
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Practice  &  Feedback 

Individual  Practice  (Guided) 
Group  Practice  (Guided) 
Individual  Practice  (Independent) 
Group  Practice  (Independent) 
Group  Feedback 
Peer  Feedback 
Delayed  Feedback 
Simulation 
Drill 

Brainstorming 
Role  Play 
Q  &  A 
Debriefing 

Closure 
Reflection 
Student  Self-Test 
Post-Test 

Teacher  Summary 
Learner  Summary 

Other 

Administration 
Assign  Roles  or  Tasks 
Ice  Breaker 
Social 
Break 

Rearrange  Classroom 
Planning  Time 
Session  Evaluation 
Other  (Instructional) 
Other  (Non-Instructional) 


Appendix  B:  PLANalyst  Evaluation  Form 


Your  Name  (optional)   

Years  of  teaching  experience  ?  Subject/Grade  level?   

Thanks  for  giving  PLANalyst  a  try.  Your  opinions  about  PLANalyst  are  very  valuable  to  me 
as  I  refine  its  features  in  preparation  for  publication.  Please  give  your  thoughtful,  frank  and 
honest  responses  to  the  questions  on  this  page.      —  Bernie  Dodge 

For  each  statement,  please  circle  the  response  that  corresponds  most  closely  with  your 
opinion. 


SD  =  Strongly  Disagree,  D  =  Disagree,  N  =  Neutral,  A  =  Agree,  SA  =  Strongly  Agree 


PLANalyst  will  be  useful  for  everyday  lesson  planning. 

SI) 

I) 

.  N  ~ 

A 

SA 

PLANalyst  will  be  useful  primarily  for  planning  special  or 

SD 

1) 

N 

A 

SA 

unusual  lessons. 

The  Rearrange  Events  feature  is  useful 

"si>" 

_ . . 

~N~ 

A  ~ 

~SA 

The  Brainstorming  feature  is  useful. 

S!) 

I) 

N 

.  A 

SA 

The  jjrogram  seemed  to  run  slowly. 

Si) 

1) 

N 

A 

SA 

PLANalyst  was  easy  to  use. 

SI) 

.  1) 

N 

A 

SA 

The  program  was  cumbersome. 

SD 

J) 

N 

A  . 

'  JS/L. ..  _ 

It  was  difficult  to  find  an  appropriate  category  to  describe 
an 

event  1  wanted  to  include. 

SI) 

D 

N 

A 

SA 

The  advice  from  Anna  was  useful  and  appropriate. 

SD 

D 

N 

A 

SA 

I'd  use  the  "Lite"  mode  more  often  than  the  "Full"  mode. 

SD 

D 

N 

A 

SA 

PLANalyst  is  too  time-consuming  to  be  practical. 

SD 

D 

N 

A 

SA 

There  were  enough  different  print  out  formats  provided. 

SD 

D 

N 

A 

SA 

The  pri  ltouts  were  legible  and  attractive. 

SD 

D 

N 

A 

SA 

The  printouts  took  up  too  much  paper. 

SD 

D 

N 

A 

SA 

In  general,  a  lesson  plan  should  fit  on  one  page. 

SD 

D 

N 

A 

SA 

The  type  on  the  printouts  was  too  small. 

SD 

D 

N 

A 

SA 

The  time  spent  putting  a  lesson  into  PLANalyst  form  would 

SD 

D 

N 

A 

SA 

be  well  spent. 

Can  you  think  of  any  categories  of  event  types  that  should  be  added  to  the  pop-up  list?  If  so, 
please  describe  them. 


Can  you  think  of  other  features  you'd  like  to  see  added? 


Did  you  experience  any  'bugs'  or  flaky  program  behavior?  If  so,  please  describe. 


If  you  had  to  summarize  your  opinion  of  PLANalyst  in  just  a  few  sentences,  what  would  you 

say? 
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Homer's  epic  poem,  K  The  Odyssey",  recounts  the  wanderings  of  the  Ithacan 
king,  Odysseus  in  his  quest  for  the  purpose  of  life.  The  Odyssey  Project  is  so  named 
because  it  is  based  on  the  philosophy  that  a  new  generation  of  America  i  schools 
must  be  viewed  as  a  dynamic  lifelong  quest  for  ever  changing  star,  iards  of 
excellence. 

This  project  is  one  of  eleven  designs  for  new  American  schools  fund  id  by  the 
New  American  Schools  Development  Corporation,  a  private  organization  which  was 
conceived  in  the  national  "America  2000"  education  strategy.  It  is  located  in  the 
Gaston  County  School  District  in  North  Carolina.  This  presentation  provides  an 
overview  of  the  elements  and  special  features  of  the  proposed  design.  It  focuses  on 
the  project's  quest  to  rebuild  the  culture  in  which  schooling  exists.  Attention  is  given 
to  the  barriers  and  challenges  which  result  from  the  uncertainty  of  change.  Public 
relations  materials  included  in  this  paper  were  developed  by  the  Gaston  County 
Design  Team. 

Events  Leading  to  Odyssey 

October,  1989  -  President  Bush  and  the  nations  50  Governors  met  in  a 
historic  summit  at  the  University  of  Virginia  to  discuss  how  America  could  improve 
education  and,  in  turn,  its  economic  competitiveness. 

February,  1990  -  The  fifty  Governors  and  the  President  unanimously 
established  six  national  education  goals  for  the  nation  to  reach  by  the  year  2000. 
These  goals  are: 

All  children  in  America  will  start  school  ready  to  learn. 

The  high  school  graduation  rate  will  increase  to  at  least  90  percent. 

American  students  will  leave  grades  four,  eight,  and 
twelve  having  demonstrated  competency  in  English, 
mathematics,  science,  history,  and  geography;  and  every 
school  in  America  will  ensure  that  all  students  learn  to  use 
their  minds  well  so  they  may  be  prepared  for  responsible 
citizenship,  further  learning,  and  productive  employment  in 
our  modern  economy. 

U.S.  students  will  be  the  first  in  the  world  in  science 
and  mathematics  achievement. 

Every  adult  American  will  be  literate  and  will  possess 
the  knowledge  and  skills  necessary  to  compete  in  a  world 
economy  and  exercise  the  rights  and  responsibilities  of 
citizenship. 

Every  school  in  America  will  be  free  of  drugs  and  violence  and  will 
offer  a  disciplined  environment  conducive  to  learning. 
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Another  component  of  the  America  2000  program  established  national  models  for 
public  education  in  school  systems  throughout  the  nation.  President  Bush 
encouraged  leaders  from  the  private  sector  to  donate  funds  to  be  used  for  this 
purpose.  All  of  the  programs  to  be  developed  were  to  move  the  nation  closer  to 
meeting  the  six  national  goals. 

July,  1991  -  A  non-profit  corporation  was  formed  to  accept  contributions  and 
a  Board  of  Directors  composed  of  some  of  America's  most  outstanding  corporate 
leaders  was  appointed.  The  Board  of  Directors  include:  John  Clendenin,  Bell  South; 
Ray  Whitman,  Eastman  Kodak;  James  Jones,  American  Stock  Exchange;  Gerald 
Bailes,  former  Governor  of  Virginia;  Louis  Grestner,  Jr.,  Nabisco;  Frank  Shrantz,  the 
Boeing  Corporation;  Ann  McLaughlin,  former  Secretary  of  Labor.  These  individuals, 
and  others  helped  secure  contributions  from  such  corporations  and  organizations  as 
Martin  Marietta,  IBM,  Ford,  AT&T,  Chrysler,  Exxon  and  the  NFL. 

September  -  November,  1991  -  Meetings  were  held  across  America  to 
explain  how  schools  would  be  selected  to  develop  these  models  of  excellence.  A  team 
of  Gaston  County  educators  prepared  one  of  the  hundreds  of  grant  applications  that 
were  submitted.  Those  team  members  were: 

Joseph  Miller,  Director 

Edwin  L.  West,  Jr.,  Superintendent 

Jane  Buckner,  Instructional  Specialist  in  Early  Childhood  Education,  Gaston 
County  Schools 

Leon  Moretz,  Director  of  Elementary  Education,  Gaston  County  Schools 
Brenda  Ratchford,  Director  of  Middle  School  Education,  Gaston  County  Schools 
Don  Ratchford,  Director  of  Secondary  School  Education,  Gaston  County  Schools 
Melinda  Ratchford,  Director  of  Media  and  Technology,  Gaston  County  Schools 
Tim  Rickman,  Principal  of  Stanley  Junior  High  School 
Vivian  Taylor,  Director  of  Student  Services,  Gaston  County  Schools 

Robert  Tyndall,  Dean  of  the  School  of  Education,  University  of  North 

Carolina  at  Wilmington 
Elizabeth  Willis,  Instructional  Specialist  for  Student  Services,  Gaston  County  Schools 

These  individuals  began  writing  the  project  proposal  in  November,  1991  and 
completed  the  application  in  January,  1992. 

February,  1992  -  The  Board  of  Education  in  workshop  session  received  and 
discussed  the  project  and  officially  voted  to  submit  the  project  at  the  regular 
February  meeting. 

June,  1992  -  Gaston  County  officials  were  invited  to  meet  with  representatives  of 
the  New  American  Schools  Development  Corporation  to  discuss  the  project  proposal. 

July  7,  1992  -  Gaston  County  was  named  a  finalist  for  the  award. 

July  9,  1992  •  The  award  was  made  public.  (Gaston  County  School  District  was  one 
of  approximately  700  proposals  submitted,  and  one  of  11  selected.) 

July  -  September,  1992  •  Negotiations  were  conducted  regarding  budget. 
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November,  1992  *  The  Gaston  County  Board  of  Education  approved  the  contract. 


Project  Overview 


The  Odyssey  Project  describes  a  formal  system  of  basic  schooling  for  students  ages  3 
to  18  with  a  developmental  prenatal  to  age  3  component.  The  project  will  use  an 
outcome-based  education  model  that  focuses  on  the  knowledge,  skills,  and  attitudes 
that  students  should  possess  when  they  graduate  from  Odyssey  learning  centers. 
Five  basic  exit  outcomes  will  provide  the  focus  for  the  instructional  program  proposed 
by  the  project:  (1)  Communicator;  (2)  Collaborator;  (3)  Creative  Producer;  (4)  Critical 
Thinker;  (5)  Concerned  and  Confident  Citizen.  Odyssey  schools  will  subscribe  to  the 
theory  of  multiple  intelligences  and  will  seek  to  develop  each  student's  unique 
talents.  At  each  level  of  schooling  diagnostic  criteria  and  specific  performance 
outcomes  will  be  designed  to  address  each  of  at  least  seven  distinct  intelligences. 

Performance  outcomes  set  for  each  level  of  Odyssey  schooling  will  include  substantive 
assessment  of  each  student's  mastery  of  English,  social  studies  (including 
geography),  science,  and  mathematics  objectives.  The  curriculum  in  these  areas  will 
reflect  an  appropriate  balance  between  learning  concepts,  acquiring  information,  and 
applying  essential  processes.  Each  Odyssey  student  will  study  a  second  language, 
music,  art,  drama,  and  kinesthetics  a  each  level  of  schooling.  The  computer  will  be 
used  as  a  basic  tool  for  instruction  «.  nd  management  in  all  the  disciplines.  The 
curriculum  will  emphasize  critical  thinking  and  problem  solving,  and  students  will 
examine  relevant  ethical  issues  and  learn  how  to  make  sound  choices.  In  all 
curriculum  areas  an  emphasis  will  be  placed  on  understanding  global  issues. 

In  addition  to  formulating  and  refining  the  exit  outcomes  set  for  Odyssey  schools,  a 
set  of  world  class  standards  will  be  designed  against  which  Odyssey  graduates  will 
be  judged.  The  set  of  world  class  standards  will  include  recommendations  made  by 
various  learning  societies  that  impact  national  and  international  curriculum 
standards.  Also  included  in  the  standards  will  be  recommendations  from  the  U.S. 
Department  of  Education  and  information  gleaned  from  top  developmental 
programs  in  the  U.S.  and  in  non-U.S.  countries. 

In  the  five  levels  of  schooling  proposed  by  The  Odyssey  Project,  traditional 
designations  of  grade  levels  will  not  be  used,  rather  a  Greek  letter  will  designate  an 
age  range  for  learners  who  attend  a  particular  center.  "Alpha"  will  designate  the  pre- 
formal  component  of  schooling  for  children  ages  0  -  3;  "Beta"  will  be  the  center  for 
learners  ages  3  -  6;  "Gamma"  for  learners  ages  7  -  10;  "Delta"  for  learners  ages  11  - 
14;  and  "Odyssey"  for  learners  ages  15  -  18.  At  all  levels  movement  in  and  out  of 
learning  cadres  will  be  determined  by  accomplishment  of  performance  outcomes  not 
by  age,  grade  levels,  or  time  frames. 

All  learning  centers  will  have  common  characteristics.  All  will  operate  on  a  year 
round  schedule.  Four  terms  of  ten-weeks  each  will  comprise  the  learning  year, 
ljcarnera  will  attend  each  of  the  four  terms  for  a  total  of  200  days  each  year.  At  the 
end  of  each  term,  a  three-week  mini-term  will  be  provided.  Learners  who  accomplish 
all  performance  outcomes  during  the  regular  ten-week  term  may  attend  enrichment 
or  extenmon  HciirtionM  or  may  take  leave  time  from  their  schooling.  Learners  who  do 
not  accomplish  all  performance  outcomes  will  attend  a  five  to  seven  day  mini-term 
that  extend*  their  leurning  time. 
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All  centers  will  provide  learners  with  high  technology  environments.  Learners  will 
leave  Odyssey  schooling  technologically  literate.  A  wide  range  of  technologies  will  be 
used  to  enhance  learning  and  manage  instruction. 

Centers  will  contract  with  many  different  agencies  to  provide  health  care,  social 
services,  wellness  and  other  essential  support  programs.  Many  of  the  agencies  will 
be  housed  at  the  centers.  This  interagency  arrangement  will  provide  holistic  support 
services  to  learners  and  their  families.  Also,  these  agencies  will  work  closely  with 
each  center's  staff  to  establish  and  maintain  a  non-violent,  drug-free  learning 
environment. 

All  centers  will  use  Paideia  concepts  as  the  primary  instructional  delivery  system. 
The  appropriateness  of  the  three  levels  of  instruction-  didactic,  coaching,  and 
seminars-will  be  understood,  properly  balanced,  and  consistently  used  during  basic 
schooling.  At  the  four  formal  Odyssey  levels-Beta,  Gamma,  Delta,  and  Odyssey- 
learners  will  stay  with  the  same  cadre  of  learners  and  facilitators  for  CORE 
learnings  during  their  years  at  each  center.  During  CORE-PLUS  time  learners  from 
different  cadres  will  work  together. 

At  all  centers  virtually  all  non-instructional  support  services  will  be  contracted.  The 
major  responsibility  of  center  professional  staff  will  be  learning  and  other  services 
will  be  managed  and  provided  by  outside  agencies.  For  all  students  at  all  levels  a 
Learning  Support  Center  will  be  provided.  Located  in  proximity  to  the  information 
center,  the  LSC  will  play  a  key  role  in  learners'  accomplishment  of  performance 
outcomes. 

At  the  three  upper  levels  of  Odyssey  schooling,  learners  will  be  required  to  attend 
weel.'y  learning  seminars  that  address  national  and  world  citizenship  ideals  and 
values.  These  seminars  will  focus  on  multicultural  issues  that  prepare  learners  for 
living  in  the  global  society  of  the  twenty-first  century. 

At  all  levels  of  formal  learning,  community  service  will  be  an  integral  part  of  each 
learner's  experience.  By  the  time  learners  graduate  from  Odyssey  Center,  they  will 
have  performed  at  least  220  hours  of  quality  community  service  time.  Family 
involvement  with  their  child's  schooling  will  be  a  primary  goal  of  The  Odyssey 
Project.  Family  members  will  provide  service  hours  at  each  Odyssey  center  and  will 
attend  a  minimum  number  of  progress  conferences  with  center  professionals  each 
year  their  child  attends  the  center. 

Business  and  community  partnerships  will  be  an  integral  part  of  Odyssey  schooling. 
A  business-e^  rtion  consortium  will  be  formed  to  determine  the  best  approach  to 
involving  businesses  in  the  basic  schooling  of  Odyssey  learners.  Each  center  will  have 
an  instructional  manager  and  a  non -instructional  manager.  The  instructional 
manager  will  be  responsible  for  the  instructional  leadership  of  the  center.  The  non- 
instructional  manager's  responsibilities  will  be  similar  to  that  of  an  operations 
manager. 

The  Odyssey  Project  not  only  breaks  the  mold  of  traditions]  schooling,  but  it  also 
breaks  the  mold  of  traditional  thinking  about  school  governance.  Constraints  to 
implementing  the  project  will  be  addressed  within  the  hurt  school  system  by  helping 
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local  board  of  education  members  understand  the  nature  of  the  project  and  to  adjust 
or  waive  constraining  policies  and  regulations.  At  state-level,  existing  and  pending 
legislation  will  be  sought  to  enable  the  project's  successful  implementation.  The 
project  can  be  fully  implemented  without  violating  federal  program  regulations. 

The  Odyssey  Project  requires  teacher  training  programs  that  are  solution  centered, 
with  theory  and  practice  integrated  through  classroom  application.  The  training  will 
be  a  joit  effort  of  university  instructors,  school  unit  specialists,  and  consultants 
from  public  and  private  sectors.  The  most  promising  educational  research  and 
programs  were  incorporated  into  the  Odyssey  Project  design.  A  synergistic 
adaptation  of  the  best  components  of  the  research  resulted  from  the  design  team's 
work. 

The  Uncertainty  of  Change:  Barriers  and  Challenges 

In  summary,  then,  Odyssey  is  best  teaching  practices,  training  for  teachers,  year 
round  school,  voluntary  early  childhood  programs,  voluntary  family  services,  limited 
multi-age  grouping,  extended  time  for  learning,  community  service,  family 
involvement,  and  expanded  use  of  technology. 

Despite  the  excellence  and  effectiveness  of  these  educational  practices,  the  lessons  to 
be  learned  from  the  Odyssey  Project  are  derived  from  the  multifaceted,  complex 
nature  of  change.  The  simultaneous  implementation  of  an  array  of  innovations  in 
pract;~es,  policies,  and  procedures  raises  questions  and  concerns  among  the 
consumers  of  the  reform.  Change  is  a  process  fraught  with  uncertainty.  It  is  a  search 
for  solutions,  new  meanings,  new  structures,  and  new  systems.  It  is,  in  fact,  a 
journey,  an  odyssey  in  the  true  sense. 

The  barriers  faced  by  the  Odyssey  Project  included: 

Legal  aspects  of  state  regulations,  legislative 
mandates,  and  the  demands  of  professional  associations. 

Lack  of  a  common  language  among  the  various 

publics. 

School  and  community  awareness  and  support. 
Opposition  from  extremist  and  special  interest 

groups. 

*         Lack  of  clarity  in  defining  "world  class"  standards. 

These  challenges  were  approached  and  met  with  varying  levels  of  success.  For 
example,  special  legislation  suspended  state  regulation*  for  schools  involved  in  the 
project.  Efforts  were  also  in  progress  for  defining  measurable  standards  and  for 
building  community  awareness  through  a  broad  public  relations  program  (  see 
Appendix  A:  Facts  About  Odyssey  Schools). 

However,  the  key  issues  were  primarily  ownership  ami  understanding.  Opposition 
from  very  vocal,  militant  special  interest  groups  undermined  the  well-designed  public 
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relations  effort  by  focusing  on  distorted  definitions  of  innovative  practices  in  the 
design.  For  example,  critical  thinking  was  interpreted  as  learning  to  criticize  parents 
and  question  family  values;  decision- making  skills  were  considered  inappropriate 
because  right  and  wrong  are  absolute;  and  global  education  was  perceived  as  an 
attempt  to  eliminate  patriotism.  These  are  fighting  words  in  the  Bible  Belt!  So.  as 
understanding  became  clouded,  ownership  wavered  and  weakened  considerably 
even  to  the  point  of  loss  of  continued  NASDC  funding.  Reform  is  truly  a  political  as 
well  as  educational  process. 

Will  the  quest  continue?  Probably  —  but  most  likely  as  pockets  of  change  instead  uf 
systemic  reform.  Dedicated  teachers,  parents,  and  administrators  at  individual 
schools  remain  committed  to  the  Odyssey  principles  and  plan  to  implement  them  at 
their  sites.  The  district  has  also  taken  a  supportive  and  nurturing  stance.  The 
university  partner  is  moving  ahead  with  the  design  of  alternative  teacher 
preparation  programs. 

Solutions  for  educational  reform  are  not  easy.  The  system  is  complex  and  hence  its 
reform  will  also  be  complex. 
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Appendix  A:  Pacts  About  Odyssey  Schools 

Q.  Why  were  school  sites  identified  before  the  planning  phase  was  completed? 

A.  Because  teachers,  parents  and  students  are  required  to  be  involved  in  helping  make 
decisions  about  the  direction  of  the  project. 

Q.  What  is  the  curriculum  that  will  be  used  in  the  Odyssey  schools? 

A.  It  is  the  same  curriculum  now  being  taught  in  the  Gaston  County  Schools-the  North 
Carolina  Standard  Course  of  Study  and  the  current  Gaston  County  Curriculum. 

Q.  Who  will  teach  in  the  Odyssey  schools? 

A.  Teachers  in  the  Odyssey  schools  will  be  experienced  Gaston  County  teachers-  Those 
currently  teaching  in  the  selected  sites  will  be  the  Odyssey  staff  unless  the-1  -4r*P 
otherwise. 


Q.  Who  developed  the  Odyssey  proposal? 

A.  The  people  who  wrote  the  entire  proposal  are  educators  from  Gaston  County  who 
have  long  careers  as  teachers  and  administrators  in  this  school  system. 

Q.  Who  is  funding  the  project? 

A.  Odyssey  is  funded  by  the  new  American  Schools  Development  Corporation  (NASDC), 
whose  board  of  directors  includes  the  CEOs  of  Exxon,  AT&T,  B.F.  Goodrich, 
BellSouth  and  Martin  Marietta,  to  name  a  tew.  NASDC  is  part  of  President  Bush's 
America  2000  program.  It  is  a  private  non-profit,  tax-exempt  organization  formed  by 
American  business  leaders  to  support  the  design  and  establishment  of  new  high 
performance  learning  environments  that  communities  can  use  to  transform  their 
schools. 


Q.  Is  the  project  connected  with  any  other  organization? 

A.  Odyssey  is  not  affiliated  with  any  other  program  or  organization,  educational  or 
otherwise. 


Q.  Where  will  Odyssey  be  located? 

A.  North  Gaston,  W.C.  Friday  and  Carr  are  the  designated  sites  for  Odyssey.  Other 
Gaston  County  schools,  however,  will  benefit  through  staff  training  and  new 
equipment. 
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Q.  What  is  Outcomes-Based  Education? 


A.  Outcomes-Based  Education  identifies  the  basic  skills  and  knowledge  that  students 
should  learn.  It  is  based  on  the  belief  that  all  students  can  learn  and  be  successful. 
It  is  a  proven  educational  strategy  that  has  been  in  practice  for  more  than  20  years. 
Six  North  Carolina  school  systems,  funded  by  the  Legislature,  are  officially  using  the 
concept  with  success  and  many  others  are  just  beginning. 


Q.  What  are  some  of  the  teaching  strategies  that  will  be  used  in  the  three  selected 
schools? 

A.  The  techniques  and  practices  to  be  used  in  Odyssey  are  based  on  strategies  that  have 
proven  successful  in  helping  all  students  learn.  Team  teaching,  a  200  day  school 
year,  and  multi-age  grouping  are  examples  of  proven  strategies  that  Odyssey  will 
use  to  enhance  learning  for  all  students. 


Q.  What  about  technology  in  the  Odyssey  classrooms? 

A.  The  students  will  learn  in  high  technology  classrooms  which  will  prepare  them  for 
21st  Century  jobs  and  for  life-long  learning. 

Q.  Will  students  have  the  traditional  grades,  class  rank,  transcripts,  and  test  scores 
required  for  college  entrance? 

A.  Yes,  In  addition,  students  will  have  a  portfolio  containing  examples  of  their  academic 
work. 


Q.  What  kind  of  learning  materials  will  be  used? 

A.  In  addition  to  textbooks,  students  will  have  unprecedented  access  to  computers  and 
software  programs.  Hookups  with  state  and  national  libraries  are  anticipated. 

Q.  What  is  the  length  of  the  school  day? 

A.  The  required  school  day  for  students  will  be  9:00  a.m.  to  3:00  p.m.  However,  the 
schools  will  be  open  from  6.30  a.m.  to  6:30  p.m.  This  schedule  will  make  it  possible 
to  extend  time  for  learning  opportunities  and  may  even  include  before  and  after 
school  care  should  parents  want  this  service  provided  at  their  school. 


Q.  What  about  graduation? 

A.  There  will  be  graduation  ceremonies  in  June  for  seniors.  There  will  also  be 
opportunities  for  some  students  to  graduate  early  if  they  choose. 
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Q.  What  will  the  school  calendar  be? 


A.  Students  will  attend  school  for  4  ten-week  terms  and  will  have  the  option  to  attend  a 
5-7  day  mini-term  or  take  3  weeks  vacation  at  the  end  of  each  term.  The  optional 
mini-term  will  help  students  catch  up  or  move  ahead  with  their  studies.  Traditional 
holidays  will  be  built  into  the  calendar. 


Q.  Will  students  still  take  achievement  tests  required  by  the  state? 
A.  Yes, 


Q.  Will  Vocational  Education/Tech  Prep  be  offered? 

A.  Yes.  All  students  will  have  access  to  a  range  of  opportunities  including  internships 
and  on-the-job  work  experience. 


Q.  Will  students  continue  to  be  eligible  for  Governor's  School? 
A.  Yes. 


Q.  Will  students  who  come  into  the  Odyssey  program  from  other  schools  after  the 
program  is  in  operation  be  at  a  disadvantage? 

A.  No.  These  students  will  be  placed  in  appropriate  levels  of  instruction. 


Q.  What  about  sports? 

A.  Odyssey  schools  will  retain  their  athletic  programs  and  coaches  will  retain  their  jobs. 
Odyssey  schools  will  continue  to  have  the  same  sports  in  the  same  conferences  in  the 
same  manner.  Some  of  the  burdens  of  managing  the  sports  programs  such  as 
collecting  gate  receipts,  providing  security  for  games,  and  providing  field  maintenance 
could  possibly  be  performed  by  booster  clubs  or  oi-ier  civic  organizations  through 
contractual  agreements. 


Q.  What  about  band  programs? 

A.  Odyssey  schools  will  retain  their  band  programs  and  band  staff  will  retain  their  jobs. 
Music  will  continue  to  be  an  integral  part  of  both  academic  and  extra-curricular 
activities. 


Q.  What  about  clubs,  student  government,  and  senior  prom? 
A.  All  of  those  school  activities  will  continue. 
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Q.  How  will  student -teacher  ratios  be  changed? 

A,  The  ratios  will  be  improved  as  compared  with  current  ratios  required  by  the  state. 
Q.  Will  ther*1  be  services  for  exceptional  children? 

A.  Public  Law  94-142  requires  that  needs  of  jxceptional  children  be  met  in  an 
appropriate  manner. 
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ABSTRACT 

Console-operation  skill  is  procedural  knowledge*  ot  control  panel  fiction*.  The  console 
operator  must  select  appropriate  sequences  of  stops  fe.g.  setting  dials,  pfenning  buttons, 
etc.)  as  mandated  by  desired  goals  and  subgoaln.  In  tlu»  I  ruining  of  console-operation  skill, 
a  Computer-Based  Training  (CBT)  environment  i'iiii  simulate  control  panel  responses  and 
device  actions  while  providing  instructional  support.  The  instructional  support  delivered 
during  practice  consists  of  a  set  of  intervention  strategies.  Thin  paper  discusses  an  ongoing 
investigation  of  CBT  approaches  in  the  acquisition  of  console-operation  skill.  Two  variables 
of  intervention  described  in  this  paper  include  the  content  of  feedback  and  the  timing  of 
feedback.  Preliminary  results  from  a  laboratory  experiment  suggest  that  feedback  which 
elaborates  on  the  goal  structure  of  the  con  Hole-operation  skill  in  more  effective  than  feedback 
that  simply  corrects  errors. 
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INTRODUCTION 

Computer-Based  Training  (CBT)  seeks  to  provide  an  environment  where  knowledge 
ii<*f|iuHitinn  ih  efficiently  and  effectively  achieved.  In  the  training  of  console-operation  skill, 
the  CUT  environment  can  simulate  control-panel  responses  and  device  actions  while 
providing  instructional  support.  This  paper  describes  an  ongoing  investigation  of  CBT 
(ipproiirheK  in  the  acquisition  of  console-operation  skill. 

At  a  basic  level,  console-operation  skill  is  the  execution  of  control-panel  actions  such 
jih  the  pressing  of  a  button  or  the  rotation  of  a  dial.  At  a  higher  leve;I,  the  skill  is  a  complex 
hierarchy  of  goals,  subgoals  and  discrete  steps.  The  console  operator  must  select 
appropriate  sequences  of  steps  as  mandated  by  the  desired  goals  and  subgoals.  During 
practice  of  this  procedural  skill,  the  instructional  support  provided  by  the  CBT  environment 
can  be  described  as  a  set  of  intervention  strategies.  These  strategies  must  resolve  two 
important  questions:  When  is  intervention  most  appropriate,  and  with  what  message  does 
one  intervene? 

Studies  of  human-tutoring  suggest  that  tutors  intervene  at  strategic  points  within 
the  execution  of  a  learning  task  (Merrill,  Reiser,  Ranney,  and  Trafton,  1992).  Tutors  often 
"overlook"  minor  errors  during  practice,  reserving  immediate  feedback  for  errors  deemed 
more  important.  In  contrast,  traditional  CBT  methods  typically  provide  immediate  feedback 
for  all  errors.  Alternative  intervention  strategies  require  sophisticated  decision-making 
processes. 

The  message  delivered  at  an  intervention  can  include  an  array  of  information  types. 
Beyond  the  simple  verification  of  incorrectness,  the  message  can  provide  an  explanation  for 
the  error,  hints  or  guidance  toward  the  correct  response,  and  even  the  correct  response  itself 
Guidance  delivered  at  an  intervention  can  direct  the  learner's  attention  to  the  goal  structure 
of  the  procedure.  This  is  often  the  approach  of  Intelligent  Tutoring  Systems  (ITS)  where  the 
content  of  an  intervention  message  goes  beyond  simple  verification  of  incorrectness.  The 
study  described  in  this  paper  attempts  to  determine  the  value  of  these  intervention 
approaches  in  terms  of  the  efficiency  of  skill  acquisition,  quality  of  achievement  and  learner 
preferences. 

The  platform  of  the  investigation  is  a  computer-based,  console-operations  simulation 
called  LOADER  (Farquhar,  1993).  Analogous  to  many  military  and  industrial  tasks  that 
call  for  knowledge  of  various  procedures,  LOADER  is  a  laboratory  research  tool  developed  to 
assess  instructional  strategies  in  the  acquisition  of  console-operation  skill.  The  task 
simulated  by  the  LOADER  environment  is  the  operation  of  a  remote  crane  control  arm  to 
load  various  canisters  from  a  set  of  storage  bins  to  one  or  more  railroad  cars  (see,  Figure  1 
for  the  simulation  interface). 

In  order  to  provide  instructional  guidance  capable  of  determining  when  to  intervene 
and  what  to  say,  an  expert  system  was  embedded  into  the  delivery  system  of  LOADER. 
The  expert  system  evaluates  error  criticality  in  oraer  to  determine  points  of  intervention.  In 
addition,  the  expert  system  analyzes  solution  methods  to  best  select  the  goal  structure 
message.  Essentially,  LOADER  behaves  as  an  Intelligent  Tutoring  System  (ITS)  when 
features  of  the  expert  system  are  employed. 

BACKGROUND 

The  evaluation  of  ITS  programs  and  methods  has  only  recently  become  a  topic  of 
discussion  in  the  ITS  literature  (Mark  &  Greer,  1993;  Murray,  1993;  Shute  &  Regian,  1993; 
Winne,  1993).  Reported  evaluations  cover  a  broad  spectrum  of  analyses  from  investigations 
of  the  internal  components  of  a  program  to  its  external  effectiveness  (Legree,  Gillis,  &  Orey, 
1993;  Littman  &  Soloway,  1988).  Internal  evaluations  focus  on  the  effects  of  specific 
components  or  processes  of  a  program,  such  as  diagnostic  accuracy  and  pedagogical 
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decisions,  while  external  evaluations  address  the  overall  educational  impact. 

In  a  review  of  ITS  external  evaluations  conducted  by  Legree  and  Gillis  (1991),  ITSs 
deemed  extensive  in  nature  (i.e.  covering  a  third  of  a  college  course  or  more  than  30  hours  of 
instruction)  showed  an  improvement  on  knowledge-based  tests  of  approximately  1.0 
standard  deviation.  This  analysis  tends  to  support  the  overall  effectiveness  of  ITSs. 
However,  the  number  of  extensive  external  evaluations  conducted  remains  very  small  (four  in 
the  Legree  and  Gillis  report).  The  1.0  standard  deviation  improvement  is  also  far  short  of 
the  2.0  •  tandard  deviation  improvement  that  appears  to  be  possible  with  human-tutoring 
interactions  (Bloom,  1984). 


Figure  1:  LOADER  interface. 


In  addition  to  conducting  more  external  evaluations,  ITS  researchers  maintain  that 
internal  evaluations  must  also  continue  in  order  to  determine  the  efficacy  of  ITS  methods. 
One  effective  ITS  methodology  supported  by  the  literature  is  model  tracing  (Anderson,  1988; 
Merrill,  Reiser,  Ranney,  &  Trafton,  1992).  A  tutor  employing  the  model-tracing  method 
matches  each  step  of  the  student's  problem-solving  behavior  to  the  possible  solution  paths. 
When  the  student's  action  deviates  from  a  known  solution  path,  the  tutor  intervenes  with 
informative  and  corrective  advice. 

The  theory  driving  the  model-tracing  methodology  is  Anderson's  ACT*  Theory 
(Anderson,  1983).  A  central  assumption  of  the  ACT*  theory  of  cognition  is  that  the  content 
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of  error-corrective  feedback  is  processed  by  learners  facilitating  their  acquisition  of 
knowledge.  In  other  words,  learners  will  gain  from  information-rich  feedback,  such  as  error 
explanations  and  goal  descriptions.  Unfortunately,  some  of  Anderson's  own  studies  fail  to 
support  this  assumption  (Corbett  &  Anderson,  1991;  Corbett,  Anderson,  &  Patterson, 

1990)  . 

The  mod  el- tracing  methodology  also  relies  heavily  on  the  use  of  immediate  feedback. 
While  the  ACT*  theory  does  not  mandate  the  use  of  immediate  feedback  (Corbett  & 
Anderson,  1991;  Corbett,  Anderson,  &  Patterson,  1990),  both  practical  and  theoretical 
reasons  exist  for  its  implementation.  Included  in  the  theoretical  reasons  is  the  reduction  of 
non-productive  floundering.  On  the  other  hand,  Schooler  and  Anderson  (1990)  have  found  a 
number  of  disadvantages  of  using  immediate  feedback:  1)  the  student  grows  dependent 
upon  feedback  and  hesitates  to  act  without  it;  2)  the  student  does  not  develop  error 
correction  and  detection  abilities;  and,  3)  the  feedback  competes  for  the  student's  limited 
short-term  memory  resources. 

.  In  a  study  designed  to  investigate  the  effects  of  immediate,  information-rich  feedback 
(Corbett  &  Anderson,  1991),  forty  undergraduates  completed  a  series  of  Lisp  programming 
exercises  with  the  CMU  Lisp  Tutor.  The  treatments  under  investigation  delivered  four 
different  types  of  feedback:  immediate  feedback,  error  flagging,  feedback  on  demand,  and  no 
feedback.   Of  the  three  treatments  that  delivered  feedback,  all  presented  similar  forms  of 
information-rich  feedback.  The  three  feedback  types  differed  primarily  in  their  method  of 
error  notification. 

Immediate  feedback,  in  the  Corbett  and  Anderson  study,  provided  information-rich 
feedback  upon  the  execution  of  every  error.  The  error  flagging  condition  notified  students  of 
an  error,  but  did  not  deliver  informative  feedback  unless  requested  by  the  student.  The 
feedback-on-demand  condition  performed  no  error  notification,  delivering  informative 
feedback  only  when  requested.  Finally,  the  no-feedback  condition  delivered  neither  error- 
notification  feedback  nor  informative  feedback. 

The  results  of  the  Corbett  and  Anderson  study  found  no  performance  differences 
between  groups  that  received  feedback.  The  no-feedback  condition  performed  consistently 
worse  on  a  number  of  performance  measures.  Essentially,  these  results  indicate  that  any 
feedback  that  serves  to  identify  errors  is  superior  to  no  feedback.  The  benefits  of  providing 
information»rich  feedback  (as  accomplished  with  model-tracing)  was  not  demonstrated.  The 
lack  of  significant  differences  between  feedback  conditions  in  this  study  can  be  explained  by 
the  small  sample  size.  Even  if  the  feedback  manipulation  was  expected  to  produce  a  "large" 
effect,  a  sample  size  of  68  would  be  required  for  an  acceptable  level  of  power  (.80)  (Keppel, 

1991)  . 

A  comparison  analysis  of  human  tutoring  and  tutoring  with  model-tracing  found  a 
few  notable  differences  (Merrill  et  al.,  1992).  Among  these  differences  is  the  use  of 
immediate  feedback.  Human  tutors,  it  appears,  are  more  flexible  in  determining  when  to 
provide  immediate  and  intervening  feedback  (Littman,  1991;  Littman,  Pinto,  &  Soloway, 
1990).  Often,  the  human  tutor  will  allow  students  to  perform  low-level  errors  without 
immediate  feedback.  Instead  the  feedback  for  such  errors  is  delivered  at  a  later  time,  often 
at  the  completion  of  the  problem.  Therefore,  human  tutors  make  strategic  decisions  in  the 
timing  of  feedback,  reserving  advice  for  the  appropriate  moment. 

Although  a  number  of  ITSs  employing  the  model-tracing  methodology  have  been 
demonstrated  as  effective  through  external  evaluations,  the  efficacy  of  their  use  of  feedback 
is  far  from  determined.  And,  since  the  content  and  timing  of  feedback  is  a  central 
component  within  these  tutors,  it  is  prudent  to  investigate  these  attributes. 

Content  of  Feedback 

The  use  of  feedback  in  widely  accepted  within  educational  theory  and  practice  as  a 
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critical  element  of  the  learning  process  (Anderson,  1983;  Bangert,  Kulik,  &  Kulik,  1983; 
Grinder  &  Nelson,  1985;  Gagne,  1984).  Extensive  research  in  feedback  has  been  conducted 
on  psychomotor  and  cognitive  skill  (Lee  &  Carnahan,  1990;  Salmoni,  Schmidt  &  Walter, 
1984;  Wulf  &  Schmidt,  1989),  testing  and  instructional  environments  (Heald,  1970;  Roper, 
1977),  tutor,  text,  and  technology-based  programs  (Kulik  &  Stock,  1989;  Schimmel,  1983), 
as  well  as  laboratory  and  field  settings.  This  review  is  limited  to  the  use  of  feedback  in  the 
acquisition  of  cognitive  skill  from  instructional  situations.  While  there  are  a  number  of 
theoretical  models  that  describe  mechanisms  of  feedback,  simply  defined,  instructional 
feedback  is  a  response  to  students1  actions. 

Actions  and  responses  to  these  actions  creates  a  cycle  of  interactivity.  A  common 
feature  of  current  theoretical  models  of  instructional  feedback  is  the  cyclic  changes  that  occur 
with  the  learner's  cognitive  state  (Bangert-Drowns,  Kulik,  Kulik,  &  Morgan,  1991;  Kulhavy 
&  Stock,  1989).  These  changes  are  influenced  by  a  number  of  factors.  Perhaps  the  most 
important  is  the  type  of  feedback  provided  (Dempsey,  Driscoll,  &  Swindell,  1993). 

Type  of  feedback  can  be  classified  into  the  following  five  categories  (Roper,  1977; 
Bangert-Drowns,  et  al.,  1991;  Dempsey  et  al.,  1993): 

1 .  No  Feedback  is  often  a  control  condition  used  in  comparison  to  other  forms  of 
feedback. 

2.  Verification  Feedback  or  Knowledge  of  Results  (KR)  Feedback  simply  informs  the 
learner  of  a  correct  or  incorrect  response. 

3 .  Correct  Response  or  Knowledge  of  Correct  Response  (KCR)  Feedback  goes  beyond 
Verification  Feedback  to  inform  the  learner  of  the  correct  answer. 

4.  Try-again  Feedback  directs  the  learner  to  make  one  or  more  additional  attempts 
at  the  answer. 

5.  Elaborative  Feedback,  may  be  a  number  of  different  forms  of  additional 
information  that  either  provides  an  explanation  for  the  error  or  reviews  relevant 
material. 

A  review  by  Kulhavy  and  Stock  (1989)  found  more  than  50  studies  completed  since 
1970  where  elaborative  feedback  was  one  of  the  variables  under  investigation.  Elaborative 
feedback,  as  defined  by  Kulhavy  and  Stock,  consists  of  any  information  delivered  to  the 
student  beyond  simple  verification.  Thus  KCR  or  Knowledge  of  Correct  Response  feedback 
would  be  included  as  a  form  of  elaboration.  Unfortunately,  the  manipulations  of  the  content 
of  elaborative  feedback  have  been  inconsistent,  making  simple  generalizations  of  results 
difficult  if  not  impossible.  In  addition,  or  perhaps  due  to  this  inconsistency,  the  results  have 
been  varied.  Kulhavy  and  Stock  cite  that  about  half  of  the  studies  they  reviewed  where  the 
manipulation  of  elaborative  feedback  consisted  of  increasing  amounts  of  information 
produced  improved  performance.  The  remaining  half  showed  no  significant  improvement. 

An  example,  cited  by  Kulhavy  and  Stock,  where  "elaborative"  feedback  was  shown  to 
be  beneficial  is  a  study  by  Roper  (1977).  In  that  study,  subjects  who  received  KCR  feedback 
within  a  computer-assisted  instructional  program  demonstrated  superior  performance  than 
those  subjects  receiving  KR  feedback.  And  KR  feedback,  in  turn,  led  to  improved 
performance  over  n  nn  feedback  condition.  Note,  however,  that  the  manipulation  in  this 
study  was  simply  KCR  over  KR  and  not  elaborative  feedback  as  defined  as  additional 
explanatory  information. 
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'*'\\v  Kulhnvy  unci  Stock  analysis  points  to  one  of  the  major  problems  with  research 
into  r Inborn ti vi*  tVu»tibnek:  a  consensual  definition.  Studies  have  included  numerous  types  of 
elaborations  from  simple  explanations  of  errors  to  re- presentations  of  the  instruction 
(Kulhavy,  1?IH5).  ()ftenl  us  with  the  Kulhavy  and  Stock  report,  mere  presentations  of  the 
correct  response  in  considered  "elaborative,\ 

Even  though  Kulhavy  and  Stock  claim  limited  evidence  for  the  benefits  of  elaborative 
feedback  (Roper,  1977),  they  conclude:  "Based  on  our  assessment  of  the  literature,  we  are 
unable  to  reach  any  usetol  conclusion  regarding  how  the  elaborative  component  of  feedback 
operates"  on  post  performance  (p.  289). 

In  a  more  recent  meta-analysis,  Bangert-Drowns,  et  al.  (1991)  also  failed  to  conclude 
that  the  amount  of  information  given  in  feedback  effects  post  performance  despite  their 
presumptions  that  "elaborate  feedback  would  have  encouraged  the  construction  of  a  richer 
network  of  pathways  to  the  desired  information"  (p.  234).  Instead,  the  major  conclusion  of 
the  Bangert -Drowns  et  al.  analysis  is  that  the  primary  role  of  instructional  feedback  is  in 
the  correction  of  errors.  In  their  report,  studies  that  investigated  the  use  of  corrective 
feedback  (elaborative  or  KCR  feedback),  while  controlling  for  major  sources  of  feedback- 
related  variation  (such  as  presearch  availability),  demonstrated  an  average  effect  size  of 
0.77.  While  supportive  of  corrective  feedback  over  verification  feedback,  the  Bangert-Drowns 
et  al.  analysis  found  little  support  for  the  use  of  information-rich  feedback. 

Only  one  study  in  the  Bangert-Drowns  report  compared  the  use  of  elaborative  or 
explanatory  feedback  to  correct-answer  or  KCR  feedback  (Sassenrath  &  Gaverick,  1965).  In 
this  study,  simple  KCR  feedback  was  found  to  be  superior  to  feedback  that  provided  an 
explanation.  A  result  inconsistent  with  an  information-processing  theory  of  learning. 

The  resulting  picture  of  the  use  of  elaborative  feedback  for  instruction  is,  at  best, 
inconsistent.  There  is  much  support  for  the  use  of  corrective  feedback,  but  beyond  providing 
corrective  information  the  role  of  information-rich  feedback  is  far  from  understood. 

Timing  of  Feedback 

A  second  major  variable  of  interest  in  the  literature  on  instructional  feedback  is  in 
timing.  An  operant  theory  approach  to  behavior  modification  insists  upon  the  delivery  of 
immediate  reinforcement  (Skinner,  1968).  This  approach  to  feedback  was  quickly 
challenged  by  what  became  known  as  the  Delay  Retention  Effect  (BrackbHl,  Bravos,  &  Starr, 
1962;  Brackbill  &  Kappy,  1962;  Brackbill,  Wagner,  &  Wilson,  1964).  Using  multiple-choice 
testing,  researchers  consistently  found  that  a  delay  in  feedback  oi  one  or  more  days 
increased  performance.  These  findings  were  generally  left  unexplained  until  Kulhavy  and 
Anderson  (1972)  proposed  the  interference-perserveration  bj-pothesis. 

The  interference-perserveration  hypothesis  suggests  that  subjects1  initial  errors 
interfered  with  their  ability  to  immediately  acquire  the  correct  response  from  feedback.  A 
delay  in  the  delivery  of  feedback  reduced  this  interference  because  subjects  essentially 
"forgot"  their  initial  (erroneous)  response.  This  hypothesis  gathered  further  support  through 
a  meta-analysis  by  Kulik  and  Kulik  (1987), 

Th«  Kulik  and  Kulik  analysis  reviewed  53  studies  that  compared  some  fo?m  of 
immediate  to  delayed  feedback.  The  studies  covered  a  broad  range  of  experiments  from 
applied  studies  with  classroom  quizzes  and  programmed  materials  to  controlled 
experiments  in  list  learning  and  learning  from  test  materials.  Such  differences  between 
studies  warranted  a  separate  analysis  for  type  of  study.  Indeed,  Kulik  and  Kulik  found 
that  immediate  feedback  was  more  effective  than  delayed  in  typical  classroom  settings  with 
an  average  effect  size  of  0.28. 

Experiments  involving  delayed  feedback  and  list  learning  have  demonstrated  highly 
equivocal  results.  Strong  support  for  immediate  feedback  as  well  as  strong  support  for 
delayed  feedback  have  been  reported.  Kulik  and  Kulik  provide  a  detailed  explanation  for 
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these  findings.  Essentially,  studies  reporting  positive  effects  for  the  use  of  delayed  feedback 
presented  feedback  in  such  a  way  that  the  feedback  constituted  a  separate  and  additional 
practice  trial  Experiments  that  controlled  for  this  event  (by  fusing  the  feedback  to  the 
instruction  through  a  reduction  in  the  time  interval  or  an  elimination  in  cues)  resulted  in  a 
greater  effect  for  immediate  feedback. 

Consistent  with  this  conclusion,  experiments  involving  the  delay  of  feedback  during 
testing  conditions  showed  gains  over  the  use  of  immediate  feedback.  Again,  the  delay  of 
feedback  in  these  studies  allowed  subjects  to  essentially  perform  additional  practice  trials. 
Since  most  of  the  studies  showing  support  for  delayed  feedback  failed  to  control  for 
additional  instructional  trials,  the  use  of  immediate  feedback  for  most  educational 
situations,  is  strongly  recommended  by  Kulik  and  Kulik. 

Unfortunately,  the  Kulik  and  Kulik  analysis  has  not  laid  to  rest  the  issue  of 
immediate  versus  delayed  instructional  feedback.  A  number  of  supporters  remain  loyal  to 
the  use  of  delayed  feedback  for  some  situations  (Dempsey,  Driscoll,  &  Swindell,  1993; 
Cohen,  1985).  Roper  (1977)  suggests  that  the  delay  of  corrective  feedback  (while  providing 
immediate  verification  feedback)  may  promote  learners  to  actively  rethink  the  problem,  thus 
giving  the  question  more  attention  than  situations  where  all  the  feedback  is  given  at  once. 
Cohen  (1985)  lists  a  number  conditions  under  which  delay  of  feedback  may  be  beneficial. 
Immediate  feedback  may  impede  the  pace  of  learning  for  high-mastery  students,  becoming 
more  of  a  distraction  than  an  aid.  Cohen  also  claims  that  delayed  feedback,  or  end-of- 
session  feedback,  facilitates  long-term  retention  and  "has  also  been  shown  to  be  helpful  if  a 
student  is  working  at  the  comprehension  and  application  level  of  Bloom's  Taxonomy11  (p.  35). 
These  claims,  however,  have  little,  if  any,  experimental  support. 

A  study  cited  by  Cohen  in  support  of  these  claims  (Gaynor,  1981),  investigated  the 
effect  of  delay  of  feedback  in  a  computer-based  mathematics  program.  Comparing 
immediate,  30-second,  end-of-session,  and  no  feedback  treatments,  the  study  analyzed  the 
acquisition  of  matrix  algebra  at  three  levels  of  performance:  knowledge,  comprehension,  and 
application.  While  noting  some  differences  of  feedback- delay  for  the  knowledge  and 
application  levels  of  the  taxonomy,  all  differences  were  statistically  insignificant.  The  only 
significant  difference  reported  by  Gaynor  were  the  post-test  scores  between  high-  and  low- 
mastery  students  -  categories  presumably  determined  by  the  subjects*  overall  performance 
on  the  task.  The  Gaynor  study,  therefore,  indicates  that  delay  of  feedback,  or  even  no 
feedback,  had  little  effect  on  performance,  A  conc!usion  also  reached  by  Mory  (1992), 

Surprisingly,  most  of  the  research  comparing  immediate  and  delayed  feedback  has 
not  been  conducted  in  the  area  of  Computer-Assisted  Instruction  (Kulik  &  Kulik,  1988). 
Instead,  applied  situations  tended  to  study  the  timing  effects  of  feedback  from  classroom 
quizzes.  The  few  CAI  studies  reviewed  by  Kulik  and  Kulik  showed  small  to  moderate  gains 
for  the  use  of  immediate  feedback. 

While  there  is  very  little  experimental  support  for  the  delay  of  feedback  ill 
instructional  situations,  there  is  some  consensus  that  delay  could  he  beneficial  in  Home 
circumstances.  One  revealing  line  of  research  in  the  use  of  instructional  teedlmek  in  in  the 
analysis  of  interactions  between  students  and  human  tutors  (Kox,  1991;  McArthur,  1990). 
As  suggested  by  Bloom  (1984),  the  effectiveness  of  human  tutoriuK  is  very  high  and  ih  often 
cited  by  ITS  researchers  as  the  ultimate  process  v<  model  (Carbonell,  1970). 

Merrill,  Reiser,  Ranney,  and  Trafton  (19f  ,)  describe  a  number  of  common  features  of 
human  tutoring  that  bring  new  insight  into  the  i,se  of  immediate  feedback  for  instruction. 
Human  tutors  maintain  a  delicate  balance  of  responsibility,  giving  students  great  amounts 
of  control  while  providing  just  enough  guidance  to  keep  them  productive.  In  doing  so,  human 
tutors  frequently  give  immediate  feedback  in  very  subtle  forms.  A  slight  hesitation  in 
responding  with  confirmatory  information  often  serves  as  a  clue  to  the  student  to  double- 
check  for  errors. 
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In  maintaining  the  role  of  a  guide,  the  tutor  is  ofttimes  presented  with  a  problem  of 
intervention.  Human  tutors  do  not  tend  to  intervene1  at  every  sign  of  error.  Instead,  the 
intervention  strategy  is  determined  by  the  context  of  the  problem.  Episodes  of  floundering  or 
critical  missteps  are  quickly  corrected,  but  feedback  on  other  types  of  errors  may  be  withheld 
until  a  more  appropriate  moment.  Therefore,  human  tutors  modulate  their  responses  based 
upon  how  serious  the  error  is  perceived  to  be.  These  attributes  of  selecting  intervention 
strategies  based  upon  the  seriousness  of  errors  has  not  been  investigated  by  researchers  of 
ITS. 

METHOD 

Independent  variables  of  the  study  consist  of  the  type  of  feedback  and  the  timing  of 
feedback  for  non-critical  errors.  Eacrfvariable  has  two  levels,  forming  a  2  by  2  matrix  (see, 
Figure  2).  The  types  of  feedback  are  instantiated  with  elaborative  and  corrective  feedback, 
whereas  the  timing  of  feedback  is  either  immediate  or  ielayed  for  non-critical  errors. 

Elaborative  feedback  includes  one  or  all  of  th  following  items:  1)  a  notification  of  an 
error;  2)  a  reason  why  the  response  was  incorrect;  3;  a  brief  description  of  the  appropriate 
subgoal;  4)  a  detailed  list  of  steps  to  complete  the  subgoal;  5)  an  indication  of  progress 
toward  completion  of  the  subgoal;  and,  6)  a  direct  identification  of  the  very  next  step. 
Corrective  feedback,  on  the  other  hand,  simply  gave  notification  of  an  error  while  identifying 
the  correct  answer. 

The  immediate  feedback  condition  delivered  feedback  (either  elaborative  or 
corrective)  upon  the  execution  of  any  type  of  error.  Conversely,  the  delayed  feedback 
condition  makes  a  decision  on  when  to  provide  feedback.  If  the  error  was  critical,  immediate 
feedback  was  given,  if  the  error  was  non-critical  (identified  as  either  inefficient  or 
inappropriate)  then  the  error  was  recorded,  but  not  immediately  reported  to  the  student. 
Under  this  condition,  all  errors  were  reported  in  an  end-of-problem  information  screen. 

Dependent  variables  of  the  study  include  time  to  complete  each  of  the  Session  1 
trials,  number  of  completed  problems  during  Session  2,  and,  speed  and  accuracy  of 
Achievement  Tests  1  and  2.  In  addition,  subjects'  ratings  of  the  treatments,  from  an  on-line 
questionnaire,  were  collected.  This  paper  includes  an  analysis  of  the  Session  1  trials  and 
the  speed  and  accuracy  of  the  first  Achievement  Test.  Data  from  the  remaining  two  sessions 
has  been  collected,  but  not  completely  analyzed  at  this  time. 

Timing  of  Feedback 


Immediate  Delayed 


Elaborative  Elaborative 
Immediate  Delayed 


Corrective  Corrective 
Immediate  Delayed 


Figure  2:  Levels  and  variables  under  study. 
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Subjects 

This  study  was  carried  out  at  the  USAF  Armstrong  Laboratory  under  the  auspices  of 
the  Cooperative  Laboratory  (CoLab).  A  total  of  119  subjects  participated  in  the  study.  The 
Air  Force  selects,  recruits,  and  hires  subjects  through  temporary  employment  agencies  to 
fulfill  experimental  laboratory  needs.  Selection  criteria  limited  subjects  to  high-school 
graduates  with  no  college  degree. 


Activity 

Data 

Session 

Tutorial  &  21  Trials 

Completion  Time 

1 

Achievement  Test  #1 

Speed  &  Accuracy 

Session 

Practice  Trials 

Completed  Problems 

2 

Achievement  Test  #2 

Speed  &  Accuracy 

Session 
3 

Summative  Evaluation 

Questionnaires 

Figure  3:  Schedule  of  Events. 

Procedure 

Groups  of  twelve  to  thirty  subjects  entered  the  laboratory  seating  themselves  at 
individual  computer  workstations.  One  of  four  treatment  groups  was  then  randomly 
assigned  to  each  subject.  After  a  brief  verbal  introduction  to  the  experiment,  subjects  began 
Session  1. 

Session  1  consisted  of  a  computer-based  tutorial  followed  by  a  set  of  21  complex 
LOADER  problems.  The  tutorial  presents  a  series  of  text  and  graphics  screens  which 
describe  the  loading  task.  Subjects  proceeded  through  this  activity  at  their  own  pace. 
Immediately  following  the  tutorial,  subjects  began  practicing  the  loading  task  through  the 
control-panel  interface  presented  in  Figure  1.  Each  treatment  was  presented  with  an 
identical  set  of  practice  problems.  The  presentation  of  practice  problems  began  with  each  of 
the  six  basic  subgoals  of  a  larger  LOADER  procedure.  The  number  of  necessary  control- 
panel  actions  for  each  subgoal  ranges  from  five  to  seven  steps.  Three  problems  of  each 
subgoal  were  presented.  The  subgoal  problems  were  followed  by  a  set  of  three  problems 
which  required  performance  of  the  complete  procedure.  Execution  of  the  complete  procedure 
is  a  32-step  process.  During  practice,  feedback  was  presented  according  to  the  treatment 
types.  Session  1  activities  ended  after  subjects  completed  an  immediate  acquisition  test. 
The  acquisition  test  was  a  32-step  LOADER  problem.  Feedback  type  during  the  acquisition 
test  was  held  constant  across  treatments.  Representative  of  the  "real"  console  panel,  the 
feedback  strategy  during  the  test  was  of  the  corrective 'delayed  type  with  no  on-line  help 
available. 

Session  2  consisted  of  subjects  practicing  additional  LOADER  problems  with  the 
tutor  for  a  period  of  2  hours.  This  practice  session  was  again  followed  by  an  immediate 
acquisition  test.  Session  3  included  a  review  and  an  evaluation  of  all  four  treatment 
conditions.  During  Session  3,  subjects  gave  each  treatment  condition  a  rating. 

LOADER:  The  Console-Operations  Task 

LOADER  is  a  complex  problem-solving  task  requiring  the  operation  of  a  simulated 
control-panel  console.  The  task  simulated  by  the  LOADER  environment  is  the  operation  of 
a  remote  crane  control  arm  to  load  various  canisters  from  a  set'of  storage  bins  to  one  or  more 
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railroad  cars.  Simple  tasks  include  the  loading  of  a  single  canister  whereas  complex  tasks 
require  the  operator  to  analyze  the  availability  and  capacity  of  railroad  cars  in  order  to 
appropriately  position  and  load  a  number  of  different-sized  canisters.  Complexity  of  the 
task  depends  upon  variables  such  as  the  number  and  size  of  canisters  to  load  and  the 
position  and  capacity  of  rail  cars.  Simple  tasks  may  be  performed  with  as  few  as  30  steps 
wnile  complex  tasks  may  require  as  many  as  150  steps. 

Each  problem  can  be  solved  through  a  set  of  six  subgoals  (or  procedures).  These 
procedures  are:  1)  Position  Rail  Car  to  Appropriate  Dock;  2)  Position  Crane  to  Appropriate 
Storage  Bin;  3)  Lift  Canister  from  Storage  Bin;  4)  Position  Crane  to  Appropriate  Rail  Car 
Receptacle;  5)  Load  Canister  into  the  Rail  Car;  and,  6)  Dispatch  Rail  Car  to  Appropriate 
Rail  Line.  Each  subgoal  has  5  to  6  individual  steps  or  control  panel  actuations.  The  rail 
yard  "situation"  determines  the  appropriate  settings  for  various  knobs  and  buttons  as  well 
as  the  correct  order  of  sub-procedures.  Complex  tasks  require  one  or  more  iterations  through 
certain  subgoals. 

The  Control-Panel  Console 

The  control-panel  console  appears  as  set  of  knobs,  switches,  buttons,  and  indicators 
appropriately  labeled  and  organized  by  function  on  the  lower  half  of  the  LOADER  interface. 
All  interactions  with  tho  console  are  performed  through  a  mouse.  While  most  interactions 
occur  with  n  nimpli*  click  of  the  mouse,  some  interactions  require  that  the  mouse  button  be 
hold  down  for  several  seconds  until  the  action  is  complete.  The  panel  responds  with  visual 
feedback  in  the  form  of  indicator  lights,  knob  and  switch  settings  with  each  selection.  For 
cert  ain  actions,  un  audible  "click"  indicates  the  completion  of  the  action. 

Tlu»  console  is  organized  into  eight  panels  associating  their  related  functions.  The 
mil  control,  line  anil  dock  paiutls  allow  the  operator  to  position  rail  cars  to  appropriate 
loading  dock*  and  dispatch  earH  to  available  rail  lines.  Operations  requiring  positioning  the 
mint'.  UHe  the?  centrally  located  panels  including  the  crane  control,  crane  coordinate  system, 
and  tho  jaws  panels,  Alohtf  tho  bottom  of  the  screen  are  panels  that  allow  storage  bin  doors 
and  mil  car  door*  to  lit?  opened. 

The  Dynamic  Mmtvl 

Located  on  tho  tipper  half  of  the  LOADER  interface  is  the  dynamic  graphical  model. 
This  model  providoH  a  representation  of  the  loading  task  from  a  birds-eye-view  perspective. 
Positions  of  the  rail  cars,  storage  bins,  canisters,  doors,  the  overhead  crane,  and  the  line  and 
dock  selections  are  indicated  with  a  number  of  graphical  elements.  Colors  aid  the 
identification  of  canister  sizes  and  rail  car  capacities. 

The  dynamic  model  responds  to  control  panel  interactions  with  animated  sequences 
representing  their  outcomes.  For  example,  engaging  a  selected  rail  car  south  with  the 
button  labeled  "South,'1  results  in  the  animated  motion  of  the  rail  car  moving  down  toward 
the  rail  docks.  In  addition  to  the  motion  of  rail  cars,  car  doors  and  bin  doors  open  and  close, 
the  crane  moves  above  the  rail  cars  and  storage  bins,  and  the  crane  lowers,  grasps  and  lifts 
canisters.  All  of  the  information  displayed  by  the  dynamic  graphical  model  (minus  the 
motion  and  graphical  relationship  qualities)  is  also  represented  on  the  upper  half  of  the 
screen  in  table  form. 

The  Expert  Help  Feature 

Embedded  within  the  interactions  of  the  problem-solving  task  is  an  expert  help 
system.  The  expert  help  provides  intelligent  advice  to  assist  in  the  solving  of  the  loading 
task.  Under  the  corrective  intelligence  mode,  expert  advice  consists  solely  of  error  notification 
while  giving  the  correct  answer.  The  elaborative  intelligence  mode  provides  levels  of  advice 
often  beginning  with  a  reason  why  the  attempted  action  was  incorrect  (see,  Error 
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Notification  in  Figure  4).  Secondly,  with  Level  1  Help,  this  mode  will  give  a  description 
of  the  subgoal  that  the  user  should  be  attempting.  By  requesting  additional  help,  the  expert 
advice  will  provide  Level  2  Help  which  is  a  list  of  the  steps  required  to  perform  the  subgoal 
with  an  indication  of  the  steps  already  accomplished.  Finally,  if  the  user  requests  still  more 
help,  the  system  will  give  the  correct  answer  (Level  3  Help).  The  levels  of  expert  help  (with 
the  exception  of  describing  the  reason  for  an  incorrect  answer)  are  also  available  to  the  user 
by  selecting  the  on-screen  Help  option. 

Other  Display  Features 

In  addition  to  the  dynumic  grnphicnl  model  mid  its  ummipunied  table,  the  LOADER 
interface  includes  a  number  of  other  helpful  display  features.  Starting  in  the  upper-left 
corner  of  the  screen,  the  name  of  the  problem  Hie  is  displayed.  Below  this,  a  verbal 
description  of  the  goal  for  the  selected  problem  appears.  Within  the  same  area,  a  Goal 
Complete  button  is  available  for  the  user  to  select  to  indicate  that  they  believe  they  have 
completed  the  problem. 


Error  Notification:  Critical  Error  -  Crane  cannot  be  lowered 
through  the  door  of  a  closed  storage  bin. 

Level  1  Help:        You  should  be  attempting  to:  Load  Canister  to 
Receptacle  2  of  Car  at  Dock  3. 

Level  2  Help:        To  Load  Canister  to  Receptacle  2  of  Car  at  Dock  3: 

*  1.  Set  West  Car  to  3 

*  2.  Set  West  Receptable  to  2 

*  3.  Enter  West  Side  Crane  Coordinates 

4.  Set  Crane  Control  to  Enable 

5.  Execute  Coordinates 

6.  Set  Crane  Control  to  Disable 

*  Denotes  Action  Complete 

Level  3  Help:       To  Set  Crane  Control  to  Enable:  Click  at  the  button 
indicated  with  the  red  arrow. 


Figure  4:  Levels  of  Expert  Help 

Software  and  Hardware  Specifics 

LOADER  is  programmed  entirely  in  OpenScript®  using  the  ToolBook®  software 
development  environment  from  Asymet  ix  Corporation.  Software  requirements  include 
MicroSoft  Windows  as  well  as  ToolBook®  run-time  files.  The  program  is  specifically 
designed  for  '486  compatible  computers  with  monitors  of  800  X  600  screen  resolution. 
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ItPHllltH 

Tlm'o  dependent  measures  are  under  investigation:  Session  1  completion  time, 
Achievement  Tent  1  mmpletion  time,  and  Achievement  Test  error  count.  Analysis  of 
iiitmnn1  with  repented  measures  on  trial  completion  times  during  Session  1  revealed  no 
Kiunil'inmt  iiitTereiires  between  treatments  [F(2,113)  =  .690,  p  =  .56],  Figure  5  shows  mean 
completion  times  uerosH  the  21  practice  trials. 
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Acquisition  Trials  (Session  1) 

Figure  5:  Mean  completion  times  of  acquisition  trials* 

Tahle  1  presents  mean  scores  for  the  achievement  test  completion  times.  Mean  error 
counts  for  the  achievement  test  are  presented  in  Table  2.  These  scores  reflect  an 
adjustment  whereby  outliers  where  removed  from  the  analysis.  Due  to  the  nature  of  the 
laboratory  setting,  where  subjects  are  paid  for  their  time,  not  their  performance,  extremely 
poor  outlying  scores  were  removed.  The  analysis  did  not  include  six  scores  for  completion 
time,  and  seven  scores  for  errors  .  ANOVAs  conducted  on  these  means  for  overall  effects 
reveal  a  significant  difference  for  errors  [F(3,108)  =  2.978,  p  =  .0347],  and  completion  time 
[K(;iv109)  =  2.709,  p  =  .0487)].  Fisher's  PLSD  test  was  used  to  determine  which  differences 
were  significant .  Of  interest  was  the  significant  difference  between  error  count  means  of  the 
elahorative-immediate  feedback  and  corrective- immediate  feedback  groups  (p  =  .0229).  All 
other  differences  between  groups  were  either  non-significant  or  of  little  interest  (such  as 
differences  between  elaborative-immediate  and  corrective-delayed  groups). 
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Feedback  Type 

Count 

Mean 

Std.  Dev. 

Std.  Err. 

Elab/lmmed 

30 

294.600 

168.164 

28.877 

Elab/Delayed 

27 

350.741 

206.678 

39.775 

Correct/immed 

28 

402.429 

196.869 

37.205 

Correct/Dolayed 

28 

448.893 

289.225 

54.658 

Table  1:  Mean  completion  time  for  Acquisition  Test  1. 


Feedback  Type 

Count 

Mean 

Std.  Dev. 

Std.  Err. 

Elab/lmmed 

30 

23.933 

25.748 

4.701 

Elab/Delayed 

26 

26.269 

26.784 

5.253 

Correct/lmmed 

29 

42.828 

33.739 

6.265 

Correct/Delayed 

27 

42.556 

38.071 

7.327 

Table  2:  Mean  errors  for  Acquisition  Test  1. 
DISCUSSION 

This  study  investigated  the  value  of  CBT  intervention  approaches  in  terms  of  the 
efficiency  of  skill  acquisition,  quality  of  achievement  and  learner  preferences.  Three  general 
hypotheses  predicted  that  the  use  of  elaborative  feedback  and  the  delay  of  feedback  for  non- 
critical  errors  would  result  in  gains  over  corrective  feedback  and  immediate  feedback  for  all 
errors.  A  discussion  for  each  of  these  hypotheses  is  now  presented. 

Hypothesis  1:  Elaboratiue  feedback  and  the  delay  of  feedback  for  non-critical  errors 
will  result  in  greater  efficiency  of  skill  acquisition.  It  was  suggested  that  feedback  of  n  more 
elaborative  nature  as  well  as  a  reduction  of  intervention  by  delaying  feedback  for  non-critical 
error*  would  result  in  greater  efficiency  of  acquisition.  No  differences  however,  worn  found 
during  the  first  session  of  training  for  any  of  the  treatment  group*,  Differences  in  acquisition 
time  may  become  evident  with  an  analysis  of  Session  2  trial  times.  Nevertheless,  with  the 
data  presented  here,  we  can  conclude  that  while  elaborative  or  delayed  feedback  may  not  bo 
more  efficient,  it  is  at  least  equivalent  to  other  strategies. 

Hypothesis  2:  Elaboratiue  feedback  and  the  delay  of  feedback  for  non-critical  errors 
will  result  in  greater  performance  of  skill  achievement.  Another  measure  of  the  benefits  of  an 
intervention  approach  is  the  post-training  performance.  We  conjectured  that  the 
intervention  strategies  of  elaborative  feedback  and  the  delay  of  feedback  for  non-critical 
errors  would  result  in  greater  post-training  performance.  In  support  of  that  hypothesis, 
significant  differences  were  found  between  the  means  of  the  elaborative-immediate  and  the 
corrective-immediate  treatments.  All  other  differences  were  not  statistically  significant. 
Therefor,  support  is  found  for  the  use  of  elaborative  feedback  when  immediate  feedback  is 
implemented.  The  delay  of  feedback  seemed  to  affect  performance  in  a  negative  manner, 
although  this  difference  was  insignificant.  Other  differences  may  not  become  evident  until 
the  sacend  session  achievement  test  results  are  analyzed. 

Hypothesis  3:  Elaborative  feedback  and  the  delay  of  feedback  for  non-critical  errors 
will  be  preferred  by  learners.  This  data  was  gathered  following  the  second  session  of  data 
collection.  The  results  have  not  been  analyzed. 

In  summary,  we  hope  to  reveal  additional  effects  of  CBT  intervention  strategies 
through  a  continued  analysis  of  the  data  collected.  The  use  of  feedback  which  elaborates  on 
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the  goal  structure  of  the  procedure  is  effective  for  console-operation  training  as  it  reduces 
achievement  errors  rates  after  only  a  brief  training  session.  The  effects  of  using  immediate 
or  delayed  feedback  for  errors  which  are  critical  is  less  ck .  r.  r,n*ese  findings  tend  to  support 
methods  used  in  many  Intelligent  Tutoring  Systems  which  offer  elaborative  feedback  during 
practice  in  a  variety  of  training  programs. 
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Abstract 

The  purpose  of  this  presentation  is  to  demonstrate  a  new  video  evaluation 
instrument  designed  specifically  for  distance  education.  It  was  designed  to  be  used  for 
instructional  design  consultation,  distance  education  teacher  training,  or  research. 
Categories  include  students  interacting  with  the  teacher,  other  students,  and  content. 

Instructional  designers  in  distance  education  are  not  always  developing  instruction, 
they  are  facilitating  the  transfer  of  teachers  from  a  traditional  to  a  high  tech  setting.  In  a 
two-way  television  setting,  teachers  accustomed  to  the  rich  environment  of  subtle 
communication  through  body  language  may  find  themselves  stymied  by  the  cavernous  feeling 
of  a  television  classroom.  Beyond  the  worries  of  the  technology  and  "being  on  TV,"  one  may 
experience  a  disconnectedness  that  makes  the  meaning  of  "distance"  even  more  pronounced. 
One  obstacle  facing  teachers  is  facilitating  interaction,  an  essential  component  to  success  in 
traditional  and  distance  classrooms  (Fulford  &  Zhang,  1993b;  Garrison,  1990).  Traditional 
methods  of  interaction  may  not  transfer  well  to  television  (Garrison,  1990;  Moore,  1989). 
Having  an  on-site  group  of  students  may  increase  the  comfort  level,  but  could  this  greater 
the  risk  of  forgetting  the  receive-site  students?  Unfortunately,  if  the  distance  learners  do  not 
perceive  acceptable  levels  of  overall  interaction,  they  are  less  satisfied  with  the  instruction 
(Fulford  &  Zhang,  1993a). 

The  linguistic  study  of  discourse  has  a  rich  past  (Searle,  1969;  Flanders,  1970).  For 
the  distance  educator,  a  linguistic  approach  may  not  be  the  most  expedient.  Moore  (1989) 
says  that  interaction  "carries  so  many  meanings  as  to  be  almost  useless  unless  specific  sub 
meanings  can  be  defined  and  generally  agreed  upon"  (p.  1).  He  provides  a  framework  for 
studying  interaction  in  distance  education  suggesting  three  distinct,  but  closely  related 
types:  learner-to-instructor,  learner-to-learner,  and  learner-to-content.  The  purpose  of  this 
research  was  to  create  an  instrument  to  quantify  and  classify  interaction  in  distance 
education  using  Moore's  (1989)  three  categories.  The  instrument  was  designed  to  be  used 
for  instructional  design  consultation,  distance  education  teacher  training,  and  research. 

Johnson  (1987)  found  that  teachers  could  be  trained  to  improve  their  ability  to 
facilitate  interaction  using  video  and  audio  tapes  to  analyze  and  categorize  their  behavior. 
He  used  Flanders'  (1970)  model  of  interaction  analysis  to  develop  the  instrument.  Flanders' 
system  of  interaction  analysis  was  defined  by  Verduin  (1970)  as  "the  systematic 
quantification  of  behavioral  acts  or  qualities  of  behavior  [sic]  acts  as  they  occur  in  some  sort 
of  spontaneous  interaction"  (p.  32).  "Teacher  talk"  is  divided  into  seven  sub  categories,  four 
that  are  indirect,  three  direct.  The  indirect  categories  are:  accepting  feelings,  praising  or 
encouraging,  accepting  or  using  ideas  of  the  student,  and  asking  questions.  The  direct 
categories  are:  lecturing,  giving  directions,  and  criticizing  or  justifying  authority.  There  are ' 
only  two  "student  talk"  sub-categories:  response  and  initiation.  "Silence"  is  a  category 
falling  outside  the  teacher  and  student  domain.  Johnson  updated  the  names  of  some 
categories,  but  the  meaning  was  essentially  the  same. 

Changing  philosophies  of  classroom  organization  are  altering  the  way  teachers 
facilitate  interaction  in  the  traditional  classroom.  Hertz-Lazarowitz  and  Shachar  (1990) 
discuss  the  differences  between  the  traditional  "whole    iss"  method  versus  a  "group 
investigation"  method.  They  said  that  "Flanders  and  1  *  colleagues  assessed  teachers' 
verbal  behavior  mostly  in  traditional  direct-instruction  classrooms"  (p.  79)  and  therefore, 
additional  categories  of  behavior  needed  to  be  derived  from  observing  cooperative 
classrooms.  Altogether,  twenty  categories  were  defined  from  transcripts  of  cooperative 
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classes:  instructing,  lecture,  short  questions  to  elicit  short  answers,  translation, 
interruption,  disciplining  one  student,  disciplining  the  whole  class,  disciplining  by  proxy, 
pluralizing,  prompting,  mediating,  mechanical  reinforcement,  competitive  reinforcement, 
spontaneous  reference  to  children's  initiatives,  helping  the  child  in  the  course  of  learning, 
encouraging  interaction  among  children,  referring  matter-of-factly  to  p-oblems  of  procedure 
and  organization,  reference  to  students'  performance,  individual  personal  reinforcement,  and 
revealing  emotions. 

Moore  (1989)  has  a  student-centered  perspective.  The  three  types  he  hus  specified 
all  originate  from  the  learner,  not  the  teacher.  Johnson's  (1987)  and  Hertz- Lazarowitz  and 
Shachar's  (1990)  instruments  are  mainly  teacher-centered.  Both  interaction  analysis 
instruments  to  evaluate  classroom  dialogue  are  detailed  and  differentiate  between  micro- 
levels  of  teacher  behavior.  This  suggests  a  major  theoretical  division  in  interaction  analysis. 
Conventional  instruments  (Johnson's  1987,  Hertz-Lazarowitz,  &  Shachar,  1990)  may  have 
been  successful  in  traditional  settings  for  research  and  training,  but,  applying  them  to  the 
distance  education  TV  classroom  is  problematic.  The  purpose  of  interaction  analysis  may  be 
different.  Johnson  (1985)  and  Hertz-Lazarowitz  and  Shachar  (1990)  focus  on  "functional" 
analysis,  while  Moore's  framework  focuses  on  the  "parties  involved".  Interaction  analysis  in 
the  traditional  classroom  is  often  for  the  purpose  of  training  new  teachers  (Johnson,  1987). 
In  many  distance  education  TV  settings,  veterans  already  have  teaching  skills,  so  the 
emphasis  is  on  adjusting  to  the  two-way  capabilities  of  the  television  classroom.  The 
dilemma  is  not  so  much  compelling  teachers  to  interact  locally,  but  to  interact  with  all 
students  across  the  distance.  Since  both  parties  are  essential  to  the  interaction  process, 
this  research  attempts  to  give  equal  care  to  students  and  teachers. 

The  idea  of  analyzing  interaction  in  two-way  television  is  valid,  but  the  instrument 
must  be  suitable  for  the  setting.  Hiring  trained  classroom  observers  is  costly.  A  workable 
tool  has  to  be  simple  to  use,  and  require  minimal  training.  Videotaping  may  be  one  *oo!  for 
evaluating  interaction.  However,  videotaping  alone  is  not  enough,  because  no  analysis  can 
be  conducted  without  a  systematic,  objective-based  approach.  Simply  watching  videos  of 
themselves  in  the  classroom  may  alienate  "technophobic"  teachers. 

The  Conceptual  Framework 

Some  may  assume  that  interaction  requires  overt  speaking  behavior.  However, 
interaction  has  also  been  defined  an  covert  behavior,  that  is  carrying  on  an  internal 
conversation  (Kruh  &  Murphy,  1990).  Therefore,  it  seems  important  to  examine  all  facets  of 
classroom  communication  in  the  study  of  interaction.  Fulford  (1993),  in  a  model  of  cognitive 
speed,  explains  that  one-way  communications  such  as  television  and  lectures  are  half  the 
speed  of  the  mind's  cognitive  capacity.  The  pace  of  one-way  communication  may  be  too  slow 
and  distract  the  learner.  Two-way  communication  requires  interaction,  keeping  the  learner's 
mind  occupied.  Normal  speech  is  usually  125-150  words  per  minute  (wpm),  but  the 
theoretical  cognitive  capacity  is  250-300  wpm.  The  model  illustrates  that  this  provides 
enough  capacity  for  the  speakers  to  simultaneously  to  speak  and  to  monitor  their  delivery, 
and  listeners  to  listen  and  to  prepare  responses.  Since  listening  requires  only  125-150 
(wpm),  "if  they  aren't  engaged  in  a  situation  in  which  they  must  interact,  their  renegade 
thought  patterns  may  dominate  their  cognitive  activity"  (Fulford  &  Zhang,  1993a).  This 
may  explain  why  anticipated  interaction  has  been  linked  to  positive  learner  attitudes 
(Yarkin-Levin  1983). 
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Flanders'  (1970)  direct  and  indirect  categories  also  seem  to  reflect  the  idea  of  one- 
way and  two-way  communication.  "Direct  influence  ...  tends  to  minimize  the  freedom  of  the 
student,  because  the  teacher  directs  the  learning  activity.  The  second  factor,  indirect 
influence,  would  have  the  opposite  effect,  or  that  of  maximizing  the  freedom  of  the  student  to 
respond"  (Verduin,  p.  32). 

For  this  study,  Moore's  (1989)  framework  of  learner- to- instructor,  learner-to-learner, 
and  learner-to-content  categories  were  sub-divided  into  "one-way"  communication  and  "two- 
way"  communication  using  Fulford's  model  (1993).  Since  interaction  may  imply  a  need  for 
overt  responses,  the  word  "communication"  was  used  to  include  both  types  of  classroom 
behavior.  One-way  communication  was  considered  a  uni -directional  information  flow  with  a 
passive  receiver.  It  was  defined  in  this  study  as  communication  directed  at  the  entire  class 
with  no  expectation  of  a  response,  such  as,  the  teacher  lecturing  or  giving  directions,  a 
student  making  a  presentation,  or  a  pre-recorded  video-tape  being  shown.  Two-way 
communication  was  considered  as  a  multi-directional  information  flow  requiring  overt 
interaction  and  active  participation  from  at  least  two  people.  It  was  defined  in  this  study  is 
directed  communication  with  expectation  of  a  response.  Similar  to  Flanders'  (1970)  indirect 
categories,  this  included:  asking  and  answering  questions,  responding  with  praise,  or 
encouraging. 

In  a  traditional  classroom,  sub-categories  of  teacher  to  student  and  student  to 
student  may  be  sufficient,  but  in  a  distance  setting  it  seemed  important  to  analyze 
interaction  across  all  sites.  Although,  Fulford  &  Zhang  (1993a)  indicate  that  every  student 
does  not  have  to  participate  publicly  to  enjoy  satisfaction  f:  om  interaction,  there  could  be  a 
group  identity  that  says  "if  my  site  is  not  called  on,  I'm  being  ignored."  For  this  reason,  two- 
way  communication  was  sub-divided  into  two  v^acher  categories  and  five  student  categories, 
The  categories  differentiated  between  the  person  initializing  the  communication  and  the 
pe?ion  retp^r  vling.  Teacher-to-atudent  and  stu^.ent-to-teacher  were  standard  categories. 
The  student -to -group  category  was  provided  in  consideration  of  collaborative  learning 
techniques  cud  to  examine  how  much  freedctn  students  have  to  converse  without  teacher 
interventior  Tea^hf.r-to-specific  location  ar)J    udent-to-specific  loepcion  were  designed  to 
examine  whether  distance  interaction  is  occurring  that  takes  into  account  grcup  identity.  A 
student-to-all  category  was  used  to  find  out  if  students  were  encouraged  to  address  everyone 
across  the  distance,  rather  than  just  the* f  ',acher.  The  student -to-content  category  was  to 
examine  how  much  active  involvement  students  ha*~e  with  the  instructional  materials.  Since 
n  large  part  of  classroom  time  could  be  taken  up  by  management  issues,  non- instructional 
categories  were  provided  for  both  one-way  and  two-way  communications. 

Flanders  (1970)  and  Johnson  (19  *7)  both  required  the  coding  of  categories  every 
three  seconds.  Skill  training,  that  lasted  several  hours,  was  supposed  to  acclimate  the  rater 
to  three  second  intervals.  There  was  no  actual  timing  device  used.  Analysis  was  often 
carried  out  through  real-time  observation,  In  this  study,  video  analysis  had  the  advantage 
til'  allowing  the  rater  to  record  the  exact  time  a  category  occurred  and  to  replay  the 
Itihtrurtion  to  bo  sure  all  cod  ng  was  accurate.  This  method  allowed  coding  of  "events"  or 
diner  ite  topics  instead  of  chopping  the  4~  -truction  into  three  second  bits.  Each  occurrence  of 
ii  rufuRoty  wan  djfined  as  an  went.  An  event  started  with  the  initiation  of  a  new  topic.  For 
example .  a  student  asked  "How  do  you  keep  the  mouse  cage  clean?"  The  teachei  responded 
"Does  anyone  have  ruggestions?"  Anothet  student  offered  "I  use  Hoil  instead  of  wood 
chip*.,."  The  teacher  praises  "Good  idea..."  When  a  third  student  asked  "What  do  you  say 
to  the  children  if  the  mouse  dies?"  this  begins  a  new  topic,  therefore  a  event. 


Analyzing  interaction  by  category  per  event  provides  a  wealth  of  information  about 
the  direction  and  participants  of  interaction.  However,  additional  information  can  be 
obtained  by  examining  single  transitions.  To  examine  the  richness  of  an  event,  it  is 
important  to  know  how  many  exchanges  occurred  between  communicators.  A  single 
exchange,  although  lasting  several  min»:U>s  is  very  different  from  numerous  exchanges  over 
the  same  period.  Coding  discrete  transactions  may  help  determine  how  many  people  were 
involved  in  the  instructional  event.  Exchanges  that  involve  only  one  teacher  and  one  student 
seem  limited,  when  tho  goal  is  to  create  lively  group  discussion,  This  study  defined 
transaction  as  the  contribution  of  a  single  individual.  A  series  of  exchanges  or  transactions 
between  individuals  constituted  an  event. 

Context  of  the  Study 

The  two-semester,  two-credit  university  course  used  in  the  study  provided  in-service 
training  for  the  Developmental  Approaches  in  Science  and  Health  (DASH)  program.  There 
were  ten  sessions  from  October  1991  to  May  1992.  DASH  is  a  sequential  kindergarten 
through  sixth  grade  (K-6)  program  that  integrates  the  content  of  science,  health,  and 
technology.  The  course  was  offered  through  the  Hawaiian  Interactive  Television  System 
(HITS)  which  is  a  4-channel  interact; /e  inter- island  closed-circuit  television  network  that 
uses  both  Instructional  Television  Fixed  Service  (ITPS)  and  point-to-point  nicrowave  signals 
to  connect  six  classrooms  across  the  state.  Instruction  was  delivered  to  five  leceive-site 
classrooms;  there  were  no  participants  at  the  origination  site,  This  was  the  first  time  this 
course  was  offered  over  HITS  and  the  first  time  these  teachers  taught  via  two-way 
television. 

Each  session  had  a  similar  format.  Participants  met  locally  for  an  hour  with  a 
DASH  facilitator,  before  the  broadcast  portion  that  lasted  one  hour  and  fifteen  minutes. 
After  a  brief  check-in,  a  pre-recorded  videotape  was  shown  for  about  twenty  minutes. 
Collaborative  activities  took  up  approximately  fifteen  minutes,  ending  with  each  location 
presenting  their  work  for  three  minutes.  The  panel  answered  faxed  questions  at  the  end  of 
the  session. 

This  course  provided  the  occasion  to  examine  a  large  number  of  participants  over  a 
long  period  of  time.  The  researchers  were  not  involved  in  the  development  or  teaching  of  the 
course.  The  videos  were  analyzed  independently  by  four  research  assistants  to  prevent 
potential  bias. 

Procedures     4  Methodology 

Th(  participants  were  K-6  teachers  who  were  already  using  DASH  in  their 
classrooms.  For  most  this  /as  their  first  interactive  TV  experience.  The  233  participants 
were  in  5  locations:  98  in  2  two-way  audio/one-way  video  locations,  and  136  in  3  two-way 
video/two- way  audio  locations.  The  10  sessions  used  in  the  study  were  recorded  at  the 
origination  site.  The  videotapes  included  only  the  broadcast  portions  of  the  course.  Due  to 
lack  of  recording  equipment,  it  was  not  possible  to  record  every  site  at  all  times.  Each  site 
was  shown  on  the  screen  as  they  participated.  During  discussion  with  one-way  video  sites, 
a  still  photograph  of  the  participants  was  shown  while  the  audio  was  heard.  Collaborative 
activities  were  the  greatest  challenge  to  videotaping.  The  two-way  video  sites  were  scanned 
in  sequence  while  open  microphones  collected  the  overall  audio  activity.  Although  this 
provided  a  4ampling  of  what  occurred  during  these  activities,  these  activities  were  rated  as  a 
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single  event  and  transactions  were  not  recorded. 

Four  graduate  students  in  educational  technology  were  recruited  a&  evaluators.  All 
of  them  had  completed  instructional  design  coursework.  They  were  given  a  description  of  the 
categories  and  then  shown  video-taped  examples.  They  were  shown  how  to  code  information 
onto  the  instrument.  The  training  lasted  about  a  half  hour.  The  evaluators  were  asked  not 
to  compare  their  evaluation  forms.  Thi?y  commented  that  the  form  was  straightforward  and 
easy  to  use.  The  four  evaluators  each  h  nalyzed  the  video  tapes  of  the  10  sessions. 

The  Evaluation  Instrument 

The  instrument  was  designed  using  Moore's  (1989)  framework  for  studying 
interaction  in  distance  education.  Three  categories  of  interaction  were  examined:  learner- 
instructor,  learner-learner,  and  learr.er-content.  The  evaluation  instrument  was  a  half  page 
form  used  to  record  each  "event"  of  the  lesson  (see  Figure  1).  An  "event"  was  defined  as  a 
single  topic.  For  example,  the  teacher  asked  a  specific  question,  a  student  answered,  the 
teacher  clarified,  and  another  student  expanded.  This  was  one  event.  If  the  teacher  asked 
a  new  question,  or  u  student  changed  the  focus,  a  new  event  began.  The  coding  of  the  event 
is  based  on  the  person  who  initializes  the  topic,  although  other  people  become  involved  in 
the  communication.  A  VCR  with  a  time-based  counter  was  used  identify  the  "beginning" 
and  "ending"  time  of  each  "event".  Both  were  recorded  on  the  form  and  a  "total"  time  was 
calculuted.  Since  ton  tapes  were  used  in  the  study,  the  tape  number  and  event  number 
were  recorded  for  tracking. 

The  type  of  interaction  was  then  recorded.  "One-way"  interaction  was  defined  as 
communication  directed  at  the  entire  class  with  no  expectation  of  response.  There  were  four 
"one«  way"  categories.  "Non-instructional"  events  pertained  to  the  management  of  the  class, 
not  the  content  of  the  lesson,  these  included  directions  about  turning  in  homework, 
purchasing  materials,  operating  equipment,  and  so  forth.  "Teacher  only"  events  were  lecture 
type  events.  "Student  only"  events  were  presentations  made  by  students.  "Content  only" 
was  for  presentations  using  media  such  as  pre-recorded  video  tapes. 


Figure  L  Video  evaluation  instrument. 
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"Two-way"  interaction  was  defined  a  directed  communication  with  the  expectation  of 
a  response.  There  were  eight  categories  of  "two-way"  interaction.  "Non-instructional"  two- 
way  differed  from  "non-instructional"  one-way  in  that  responses  were  given.  Site  sign-ons 
are  an  example  of  a  "two-way  non-instructional"  event.  "Teacher-student"  was  a  teacher 
asking  a  question  directed  to  all  students.  "Teacher-specific  location"  was  a  teacher 
directing  a  question  to  a  specific  location  or  site.  "Student-teacher"  was  the  student  asking 
a  question  directed  to  a  teacher.  "Student-specific  location"  was  a  student  directing  a 
question  to  a  specific  location  or  individual.  "Student-group"  was  a  student  convening 
within  their  own  site,  including  the  entire  group  or  groups  formed  for  collaborative  activities. 
"Student-content"  was  a  student  interacting  with  course  materials  that  required  active 
participation.  This  category  was  intended  for  written  activities,  rending,  nr  using  computer 
assisted  instruction,  although  in  this  study,  none  of  these  events  occurred.  "Student-all" 
was  when  a  student  asked  a  question  for  anyone  to  respond  to. 

Next,  individual  transactions  wert>  recorded.  This  portion  of  the  instrument  was 
designed  to  provide  an  indication  of  the  richness  of  the  communication.  For  example,  if  the 
teacher  asked  a  question,  a  student  provided  a  short  answer,  and  then  the  teacher 
expounded  for  several  minutes,  the  event  would  be  very  teacher  focused,  However,  if  the 
teacher  asked  a  question,  a  student  provided  a  short  answer,  then  the  teacher  asked  for 
elaboration,  another  student  responded,  and  another  teacher  provided  another  example,  the 
event  would  he  richer  and  more  student  focused.  These  events  may  take  the  same  amount 
of  time,  hut  by  recording  transactions,  an  instructional  designer  could  examine  patterns  of 
interaction. 

For  this  study,  transactions  were  recorded  by  using  T  for  teacher  and  S  for  student. 
If  more  than  one  teacher  or  student  was  involved  in  an  event,  numbers  were  added.  The 
recording  of  the  first  example  above  would  be  T  S  T,  the  second  example  would  be  T  S  T  S2 
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T2-  To  insure  the  accuracy  of  the  ratings,  ^valuators  were  asked  to  rewind  the  tape  and 
record  lengthy  events  three  times  (see  the  throe  lines  in  Figure  1),  The  numher  of 
transactions  was  counted.  A  filial  count  was  determined  using  the  evaluatora'  best 
judgement  which  of  the  three  attempts  was  most  accurate.  The  evaluatora  then  counted 
and  recorded  the  number  of  people  involved  in  each  even!  and  circled  the  involved  sites. 

Reliability 

The  video  interaction  analysis  instrument  was  tried  out  during  the  DASH  program. 
The  raters  viewed  the  video  tapes  of  the  ten  DASH  sessions  independently  and  recorded: 

1.  The  total  number  of  occurrences  for  each  type  of  interaction  in  each  session, 

2.  The  total  time  spent  for  each  type  of  interaction  in  each  session, 

3.  The  total  number  of  events  in  each  session. 

4.  The  total  number  of  transactions  in  each  session. 

5.  The  total  number  of  people  involved  in  all  the  events  in  each  session. 

The  information  above  was  used  to  generate  a  quantitative  summary  of  the  overt 
interaction  in  the  TV  classroom.  From  Item  1 ,  one  could  see  how  frequently  each  type  of 
interaction  occurred  in  any  session  and  determine  whether  the  interaction  pattern  over  TV 
was  balanced  or  appropriate.  Item  2  showed  the  proportion  of  time  actually  spent  on  each 
type  of  interaction,  which  revealed  the  time  reserved  for  the  particular  type  of  interaction. 
Item  3  showed  how  many  topical  aegmentH  or  events  of  teacher-student  interaction  occurred 
(Figure  2).  Item  4  showed  how  many  exchanges  or  transactions  occurred  (Figure  3). 
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Figure  2  
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Figure  3  
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 Figure  4  
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Figure  5  

Total  People 


Events 


S1       S2       S3       S4       S5       S6       S7       S8       S9      S 10 


Session 

E3  Total      □  Rater  1       ♦  Rater  2      ■  Rater  3 


i 


224 

230 


 Figure  6  
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Items  3  and  4  produced  the  average  number  of  transactions  per  event,  which  was  a 
convenient  index  of  the  extent  to  which  an  instructor  allows  a  topical  segment  to  elapse 
(Figure  4).  Item  5  showed  the  total  involvement  during  the  session  (Figure  5).  This  total 
was  derived  from  adding  the  number  of  people  in  each  event,  therefore,  the  same  individuals 
may  be  counted  a  number  of  times  to  create  this  figure.  Item  3  and  5,  on  the  other  hand, 
showed  the  average  number  of  participants  the  instructor  allowed  in  an  event  (Figure  6).  A 
highly  interactive  class  would  be  expected  to  generate  more  exchanges  and  engage  more 
students  per  event  than  a  lecture-type,  non-interactive  class. 

Aggregate  reliability  is  considered  invalid  if  the  pairwise  inter-rater  reliabilities  vary 
a  great  deal,  for  instance,  between  -  0.6  and  +  0.6  Aggregate  reliability  is  valid  only  when 
the  pairwise  inter-rater  reliabilities  are  similar  (Overall,  1965).  All  the  four  raters  provided 
the  total  number  of  occurrences  for  each  type  of  interaction  and  the  total  time  actually  spent 
on  each  type  of  interaction  in  each  session.  Their  mean  pairwise  inter-rater  reliabilities  and 
the  aggregate  reliabilities  on  the  two  variables  are  reported  in  Table  1.  The  reliabilities  are 
generally  acceptable,  except  in  those  categories  where  few  occurrences  were  observed.  Not 
all  the  categories  of  interaction  are  in  Table  1,  because  some  types  of  interaction  did  not 
take  place  in  the  DASH  program.  The  "content  only"  category  had  only  one  occurrence, 
thereby  creating  a  misleading  perfect  reliability  so  it  was  not  reported. 

Three  raters  completed  analyzing  the  events  and  transactions  in  the  video  tapes  of 
the  10  DASH  sessions.  Therefore,  the  reliabilities  reported  below  in  Table  2  were  based 
upon  three  not  four  raters.  The  reliabilities  reported  show  that  using  the  mean  of  three  or 
four  raters  will  result  in  high  reliabilities. 
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Table  1 

Reliabilities  for  total  number  of  occurrences  and 
total  time  spent  for  each  type  of  interaction 


Mean  inter-rater  Aggregate 
reliability  reliability 


Tvui!               Subcutouorv            Number         Time          Number  Time 

One-way 

Nmi -instructional 

0.38 

0.63 

0.71 

0.82 

Teacher  only 

0.89 

0.93 

0.97 

0.98 

One-way  Total 

0.60 

0.85 

0.86 

0.96 

Two-way 

Non-instructional 

0.72 

0.62 

0.91 

0.88 

Teacher  to  student 

0.39 

0.33 

0.72 

0.68 

Student  to  teacher 

0.81 

0.66 

0.94 

0.89 

Student  to  student 

0.31 

0.91 

0.64 

0.98 

Two-way  Total 

0.27 

0.97 

0.61 

0.99 

Table  2 

Reliabilities  for  total  number  of  events,  transactions,  and  people  involved, 
and  average  number  of  transactions  per  event  and  people  per  event 

Mean  Pairwise  Aggregate 
Inter-rater  Reliability  Reliability 


Number  of 
Events 

0.80 

0.93 

Number  of 
 Transactions 

0.79 

0.92 

Number  of 
People  Involved 

0.66 

0.86 

Average  Number  of 

Transaction  per  Event 

0.61 

0.82 

Average  Number  of 
PeoDle  Der  Event 

0.62 

0.83 

After  the  four  raters  had  analyzed  the  video  tapes  of  the  10  DASH  sessions, 
pair  wise  inter-rater  reliabilities  and  aggregate  reliabilities  were  calculated.  The  pair  wise 
inter-rater  reliability  is  simply  the  correlation  between  the  counts  or  recorded  times  of  any 
two  judges  (Sax,  1989).  The  aggregate  of  effective  reliability  is  the  reliability  of  the  mean  of 
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tin*  counts  or  times  independently  given  by  the  four  raters  (Guilford,  1954;  Rosenthal,  1987; 
Rosenthal  &  Rosnow,  1991).  The  aggregate  reliability  estimates  the  proportion  of  the 
variance  in  the  moan  of  the  four  scores  that  is  due  to  true  scores,  and  its  formula  follows  the 
same  logic  as  the  Spearman-Brown  prophecy  formula  (Sax,  1989,  pp.  265-266;  Rosenthal  & 
Rosnow,  1991,  pp.  51-54).  The  aggregate  reliability  is  higher  than  the  mean  of  pairwise 
intor-rater  reliabilities  for  the  simple  reason  that  employing  multiple  judges  and  adopting 
their  mean  is  a  more  reliable  scoring  procedure  than  relying  on  any  two  potentially 
idiosyncratic  individuals  serving  as  raters. 

Discussion 

Episodes  that  may  be  categorized  into  more  than  one  category  were  sometimes  a 
problem.  For  example,  the  difference  between  "non •instructional"  and  "teacher  to  student" 
was  not  always  easy  to  discern.  An  example  from  the  tape  was  a  conversation  that  began 
with  asking  about  materials  being  mailed  which  is  a  "non-instructional"  event;  ended  with  a 
discussion  of  how  the  materials  should  be  used  when  they  arrive,  a  "teacher  to  student" 
event.  One  judge  may  decide  to  categorize  the  event  at  the  beginning  of  the  transaction, 
another  at  the  end,  and  still  another  may  break  the  event  into  two  separate  events.  These 
episodes  that  do  not  seem  to  have  precise  beginnings  and  endings  may  cover  more  or  less 
time  allotment  units.  Such  ambiguity  is  more  evident  in  some  categories,  such  as  "two-way 
teacher-to-student"  and  "two-way  student- to- student".  It  is  not  clear  whether  these 
categories  are  superfluous,  or,  due  to  the  way  this  course.was  taught.  Categories  such  as 
this  could  be  grouped  together,  or  eliminated,  however,  it  is  possible  in  some  classes  the  only 
interaction  occurring  is  "non-instructional."  In  this  case,  removing  the  category  eliminates 
important  information.  The  findings  suggest  when  training  judges,  those  categories  should 
be  emphasized. 

Some  sub-categories  produced  little  or  no  data.  For  this  study,  "student  to  specific 
locution";  "student  to  group";  and  "student  to  all"  were  collapsed  into  an  overall  student  to 
student  sub-category  to  create  more  meaningful  analysis.  "Student  to  contont"  had  to  he 
eliminated  for  the  luck  of  data.  Does  it  mean  these  categories  are  superfluous?  Or,  is  it  only 
because  of  the  way  this  particular  course  was  taught?  In  the  former  case,  they  may  be 
t'liminatiiri  from  tho  instrument  or  grouped  in  order  not  to  distract  judges,  Although  the 
ration  priHTHH  in  Hitnplif it«d,  removing  categories  reduces  detailed  information. 

Till"  "content"  categories  alHO  caused  some  problems.  The  episode  of  watching  n 
videotape  was  too  obvious  to  really  test  the  extent  of  the  concurrence  among  tho  raters.  In 
the  "two-way,  student-to-content"  sub-category,  no  episodes  occurred,  "Content"  events  may 
he  lint h  difficult  to  measure  and  difficult  to  include  in  an  interactive  TV  setting.  It  may  he 
considered  a  waste  of  costly  air  time  to  show  video-tapes,  have  the  students  read,  or 
complete  written  or  computer  assisted  instruction  alone.  These  activities  can  be  done  "off- 
air"  while  collaborative  activities  and  a  discussion  of  the  results  "on-air"  may  increase  the 
amount  of  interaction  time  available,  More  research  is  needed  regarding  the  concept  of 
"interacting  with  content"  as  proposed  by  Moore  (1989). 

The  overall  reliability  of  this  instrument  is  sufficiently  high  to  warrant  its  use 
analyzing  interaction  in  distance  education,  dare  should  he  taken  not  to  just  consider  the 
amount  of  time  spent  interacting,  but  also  the  richness  and  patterns  of  interaction.  By 
using  this  instrument  as  a  consulting  tool,  instructional  designers  may  be  able  to  help 
instructors  improve  the  quality  of  interaction  across  the  distance. 
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Traditional  ID  models  lead  to  an  engineering-style  linear  planning  process  (see  Figure  1)  led 
by  technocrats  a-*d  relying  almost  solely  on  expert  analysis.  The  implications  of  the 
traditional  model  are;  training  and  performance  requirements  occur  late  in  the  process; 
training  solutions  are  often  fixes  to  other  broken  parts  of  the  system;  and  human 
performance  personnel  are  out  of  the  primary  decision  loop.  Increasingly,  people  find  this 
approach  severely  lacking  in  the  face  of  problems  that  call  for  customer/user  focus,  adaptable 
technical  and  social  designs,  and  solutions  where  training  is  impractical  or  unaffordable. 
This  brief  paper  is  intended  to  outline  an  alternative  model  based  on  the  socio-technical 
paradigm,  which  in  our  view  suggests  a  better  match  to  a  growing  list  of  problems  in  ID. 


From  a  socio-tochnical  perspective,  the  purpose  of  design  and  development  is  to  support 
human  behavior,  not  to  create  an  innovation  (Eason,  1988).  Innovations  are,  therefore, 
situated  in  settings  of  complex,  unpredictable,  multi-form  relationships  among  the  various 
elements  that  make  up  a  working  organization.  Successful  technology  is.only  one  critical 
component  among  many  that  are  highly  interactive  and  interdependent  with  each  other. 
Technology  is  only  one  component  of  what  Moran  and  Anderson  (1990)  refer  to  as  the 
"Workaday  World"  amd  Harbour  (1992)  refers  to  as  the  PETT  (People-Environment-Task- 
Technology)  model  of  organizational  processes. 

From  a  socio-technical  perspective,  organizations  are  comprised  of  people  using  technology  to 
perform  specific  work  practices  in  a  particular  environment  (see  Figure  2)  (Goodrum,  Dorsey, 
&  Schwen,  1993). 

These  four  components  can  be  briefly  defined  as  (adapted  from  Moran  and  Anderson  (1990)): 
People:  The  work  and  learning  roles  people  play,  patterns  of  formal  and  informal 
communication,  social  relationships,  and  so  on.  Environment:  The  physical  and  cultural 
aspects  of  the  organization  such  as  open  or  closed  floor  plans,  virtual  environments, 
organizational  values,  traditions,  and  so  on.  Work  Practice:  The  knowledge,  skills,  and 
routines  for  accomplishing  specific  tasks  associated  with  the  work  and  learning  roles. 
Technology:  The  technical  computation,  communication,  and  multimedia  systems  which 
support  the  other  components. 

Each  component  of  the  organization  is  seen  as  being  integrally  tied  to  the  other.  A  change  in 
one  necessarily  affects  a  change  in  the  others.  Therefore,  organizational  solutions  must  be 
planned  taking  all  into  account  (Eason,  1988).  Instructional  developers,  therefore,  need  to 
deal  with  creating  and  investigating  whole  environments,  not  just  the  technological, 
pedagogical,  or  truining  innovations  of  specific  interest. 


The  planning  process  (see  Figure  l\)  \h  a  concurrent  one  t lint  hringH  human  performance 
people,  an  well  hh  fit  hers  who  are  often  nut  of  the  decision  loop,  fully  into  the  process.  In  the 
traditional  model,  by  the  time  the  decmion  cnineH  down  to  the  training  and  performance 


(INSERT  FIGURE  1) 


(INSERT  FIGURE  2) 


(INSERT  FIGURE  ID 
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people,  the  design  my  already  be  flawed  with  no  chance  for  redesign,  requiring  the  fitting 
of  the  people  and  organization  to  the  innovation,  versus  the  innovation  fitting  the  people  and 
organization. 

Similarly,  the  socio-technical  design  team  (see  Figure  4)  includes  all  of  those  people  who 
have  an  invested  interest  in  the  success  of  the  innovation.  The  key  people  included  here  are 
the  direct  and  indirect  users  of  the  innovation.  These  are  the  people  who  must  live  with  the 
consequences  of  the  innovation  long  after  the  designers,  implementers,  and  project  managers 
have  disappeared.  Teachers  and  students  in  school  and  supervisors  and  workers  in  the 
workplace  should,  from  a  socio- technical  perspective,  be  included  from  the  very  beginning  of 
the  decision  process  as  well  as  the  innovation  design  process. 

(INSERT  FIGURE  4) 

Within  the  large  scale  development  project  there  are  many  small  scale  design  processes 
which  go  through  a  full  problem  solving  process  of  iterative  design  (see  Figure  5),  for 
purposes  of  exploring  options  and  developing  a  fuller  understanding  of  the  requirements. 

The  creation  of  prototype  versions  of  proposed  socio-technical  solutions  are  for  the  purpose  of 
evaluation  and  refinement  of  specifications.  This  allows  users  and  others  to  have  a  realistic 
experience  for  basing  assessments  and  revisions.  Conceptual  prototypes  allow  for  early  user 
reaction,  feedback,  and  projection  of  consequences.  'Working1  prototypes  allow  for  hands-on 
use  in  the  context  of  the  task.  Alternative  prototypes  in  the  early  stages  of  development 
assist  the  team  in  keeping  an  open  mind  and  not  locking  into  a  particular,  potentially 
flawed,  design  too  early. 


(INSERT  FIGURE  5) 

In  summary  to  this  point:  Development  occurs  iteratively,  allowing  for  concurrent  problem 
definition  (i.e.,  analysis),  innovation  prototype  creation,  and  evaluation.  Development  is 
conducted  by  a  cross-functional  team  comprised  of  people  who  have  a  stake  in  the  product's 
success.  Development  does  not  focus  solely  on  the  creation  of  technology.  To  the  contrary, 
the  focus  is  on  the  work  practice,  the  relationships  between  people,  the  social  and  cultural 
environment;  the  technology  should  support  the  changes  in  the  other  three  areas. 

This  process  is  not  a  one-time  occurrence.  Rather,  it  goes  through  successive  phases  (see 
Figure  6),  each  of  which  allows  for  a  certain  level  of  evaluation  as  well  as  a  projection  of 
additional  levels. 

(INSERT  FIGURE  6) 

Kach  phiiHo  can  he  described  with  a  primary  activity,  product,  and  evaluation  output. 
During  the  initial  phaHe,  a  team  brainstorms  a  vision  and  can  gather  data  on  people's 
projections  concerning  how  stakeholder**  will  react  to  the  innovation,  how  usable  the 
innovation  might  ho,  what  performance  improvements  or  behavior  changes  might  result,  and 
what  impart  that  might  have  on  the  core  business  of  the  organization  (e.g.,  profitability  of  a 
business,  teaching  and  learning  in  schools  and  universities). 

During  the  second  phase,  a  team  shows  alternative  idea  sketches  of  the  innovation  and  can 
gather  reactions  as  well  as  projections  of  the  usability,  performance  improvements  and 
bottom  line  results, 
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In  the  third  phas*;.  tbv-  ii»am  creates  opportunities  for  hands-on  use  of  one  or  more  partially 
funct:oi.inir  movk>upH<  <  hsorving  tho  innovation's  usability,  and  also  gathers  reactions  as  well 
as  projections  of  fwvHion  performance  improvements  and  bottom  line  results. 

In  tho  fmuth  phase,  tho  t(»am  pilot  tests  one  or  more  alternative  working  prototypes, 
observing  porfoi'imuu'c  improvements  or  changes  in  behavior,  gathering  additional  and  more 
refined  information  on  tho  innovation's  usability  as  well  as  users'  reactions,  and  projects 
bottom  lino  results. 

In  the  fifth  phuKC,  tho  team  fully  implements  the  completed  design  of  the  innovation, 
understanding  that  thin  may  be  only  the  first  implementation  of  an  evolving  vision.  One  can 
now  gather  data  on  bottom  line  results  on  the  core  business  of  the  organization,  while 
gathering  additional  information  on  performance  improvements,  usability,  and  users' 
reactions. 

The  scope  of  a  project  will  more  likely  increase  than  narrow  as  the  relationships  among  the 
four  components  of  the  socio-technical  system  are  better  understood  and  the 
solutions/innovations  are  capable  of  being  more  substantively  evaluated.  Alternative 
prototypes,  tested  against  one  another  during  the  early  phases,  will  combine  into  a  single 
design  for  full  implementation. 

There  is  also  an  emphasis  on  developing  for  organizational  change,  understanding  that 
solutions  must  evolve  over  time.  For  people  can  only  deal  with  so  much  change  at  one  time, 
whether  on  the  job  or  in  the  classroom.  Furthermore,  one  can  implement  revolutionary 
change,  but  the  impact  cannot  be  predicted.  Evolutionary  change  is  required  in  order  to 
have  a  chance  at  understand  the  resulting  effects. 

The  successive  phases  of  development  -  each  with  concurrent  problem  analysis, 
development  and  evaluation    become  increasingly  concrete.  The  team  may  progress 
through  phases  quickly,  depending  on  a)  the  scope  of  the  change,  b)  how  informative  the 
evaluation  outputs  are,  c)  how  closely  prototypes  fit  the  needs  of  the  individuals  and 
organization,  d)  how  much  development  from  scratch  is  required  versus  use  of  existing  or  off- 
the-shelf  materials,  e)  the  commitment  of  the  client  and  client  organization,  and  0  how  well 
consensus  is  negotiated. 

Currently,  in  our  view  the  socio-technical  approach  to  instructional  development  is  based  on 
seven  fundamental  principles: 

1)  Socio-Technical  Systems  Comprised  of  People.  Environment,  Work  Practice,  and 

Technology 

2)  A  Focus  on  Organizational  Change,  not  Technology 

3)  A  Design  Team  Consisting  of  All  (or  representatives  of  all)  Stakeholders  throughout  the 

Process 

4)  Iterative  Cycles  of  Design 

5)  Creation  of  Alternative  Prototypes 
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ti)  Concurrent  Analysis,  Creation,  and  Evaluation  Becoming  Concrete  Over  Time 
7)  Evolutionary  not  Revolutionary  Phases  of  Development 

To  (lute,  our  own  work  in  developing,  applying,  and  learning  from  a  socio- tech ni  ;al  approach 
has  included  a  number  of  projects,  ranging  from  individual  instructors  creating  lecture 
presentation  materials  to  largo  companies  in  the  process  of  changing  the  way  thousands  of 
employees  work  and  learn.  In  every  case,  the  more  we  worked  from  a  socio-technical 
paradigm,  the  better  we  were  able  to  foresee  and  resolve  many  barriers  as  well  as  make 
more  efficient  and  effective  progress  in  the  design,  creation,  and  implementation  of 
innovations.  The  socio-technical  paradigm  provides  for  ourselves  an  approach  to 
development  where  wc  can  still  apply  the  skills  and  knowledge  from  areas  such  as 
instructional  strategies,  learning  theory,  etc.,  which  in  and  of  themselves  do  not  provide  a 
framework  thot  leads  to  successful  use.  We  also  attempt  to  continuously  reflect  on  our 
mistakes  and  failures  (as  well  as  our  successes)  in  adjusting  and  refining  the  approach. 
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Abstract 


The  purpose  of  this  exploratory  study  was  to  examine  the  effectiveness  of  learner 
manipulation  of  visuals  with  and  without  organizing  cues  in  computer-based  instruction  on 
adult's  factual,  conceptual  and  problem  solving  learning.  An  instructional  unit  involving  the 
physiology  and  anatomy  of  heart  was  used.  A  posttest-only  control  group  design  was 
applied  with  74  undergraduate  subjects  from  a  large  eastern  University.  Subjects  were 
randomly  assigned  to  one  of  three  treatment  groups  -  Learner  Generated  with  Cued  Visual 
Organization,  Learner  Generated  with  Uncued  Visual  Organization  and  Control  (System- 
Provided  Organization).  The  dependent  variables  measured  the  subject's  ability  to  identify 
individual  parts  of  the  heart  using  visual  cues,  retain  facts  and  definitions,  reproduce  key 
information  and  utilize  the  information  for  problem  solving.  No  differences  were  found 
between  the  Control  and  the  Learner-Generated  Groups.  For  the  problem  solving  measures, 
the  Learner-Generated  with  Cued  Visual  Organization  group  scored  significantly  higher  than 
the  Uncued  Visual  Organization  group.  Subjects  in  the  control  group  also  scored 
significantly  higher  than  the  Uncued  Visual  Organizational  group  for  problem  solving.  In 
post  hoc  comparisons,  trends  were  found  with  the  Cued  Visual  Organizational  group  scoring 
significantly  higher  than  the  Uncued  Visual  Organizational  group  for  the  identification, 
problem  solving  and  drawing  measures. 
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INTRODUCTION 


Technological  advances  have  evolved  computer-based  instruction  into  an  increasingly 
visual  and  engaging  learning  system.  With  these  advances,  new  strategies  of  presenting 
information  within  computer-based  media  have  emerged.  The  graphical  user  interface  of 
current  computer  technology  and  development  software  allow  for  the  manipulation  and 
movement  of  visual  elements.  Providing  opportunities  for  active  learner  manipulation  of 
visual  elements  on  the  screen  may  demonstrate  and  encourage  generative  processing  of 
information.  But,  is  the  mere  manipulation  of  visuals  sufficient  to  increase  cognitive 
processing  or  should  some  type  of  cueing  strategy  be  provided  so  that  the  learner  can  actively 
construct  visuals  into  a  meaningful  arrangement?  Theoretically,  generative  learning 
strategies  support  this  notion.  Generative  learning  strategies  have  shown  that  interaction 
with  and  selective  attention  to  sensory  information  may  establish  links  to  memory  in 
constructing  meaning  (Wittrock,  1974a,  1974b,  1990). 

Manipulation  of  individual  visual  elements  may  be  more  effective  when  learners 
create  their  own  visual  organization  versus  an  organization  or  cued  visual  structure  provided 
by  the  system.  Activities  to  encourage  learner's  organization  or  ordering  of  the  information 
can  activate  the  encoding  process  which  builds  connections  between  thoughts,  develops  a 
cognitive  organization  for  ther  and  integrates  relevant  information  in  working  memory  with 
a  learner's  prior  knowledge  (Weinstein  &  Mayer,  1984). 

Empirical  evidence  to  support  the  use  of  a  learning  strategy  for  manipulating 
individual  visual  elements  onto  an  organizational  framework  in  CBI  is  lacking  in  the 
literature.   Numerous  studies  exist  involving  the  use  of  visuals  within  instruction  (e.g., 
Dwyer,  1978;  Guttman,  Levin  &  Pressley,  1977;  Levin,  1986).  However,  the  manipulation 
of  visual  elements  with  and  without  organizational  cues  in  computer-based  instruction  is  an 
instructional  strategy  only  recently  possible  through  current  technology,  and  was 
investigated  in  this  study. 

Visuals 

The  use  of  visuals  within  instruction  has  been  the  subject  of  a  large  body  of  research 
over  several  decades.  Many  studies  conducted  with  children  have  dealt  with  the  recall  of 
information  from  illustrations  that  support  aural  or  print  instruction  (e.g.,  Guttman,  Levin, 
&  Pressley  1977;  Levin,  1986).  Levin  (1989)  has  concluded  from  this  accumulated  literature 
that  pictures  presented  in  conjunction  with  text  improve  comprehension  and  memory  over 
information  presented  by  text  alone. 

Related  to  this  body  of  research,  Guttman,  Levin  &  Pressley  (1977)  conducted  a 
study  involving  oral  prose  and  visuals.  After  reading  a  story,  they  presented  visuals  with 
missing  pertinent  information  (or  partial  pictures)  to  kindergartners.  Contrasting  this 
treatment  with  the  presentation  of  the  complete  visuals,  the  experimenters  found  that  the 
incomplete  pictures  did  not  have  an  effect  on  learning.  The  presentation  of  incomplete 
visuals  devoid  of  critical  content  may  be  more  powerful  through  the  construction  of  a  visual 
organization. 

The  manipulation  and  organization  of  visuals  has  been  examined  through  the 
illustration  of  stories  with  cutout  characters.  Lesgold,  Levin,  Shimron,  and  Guttman  (1975) 
conducted  a  series  of  experiments  to  examine  the  potential  benefits  of  pictorial  elaboration 
with  young  children.  Interestingly,  the  researchers  found  that  experimenter-constructed 
illustrations  provided  significantly  improved  acquisition  of  information  in  comparison  with 
learner-constructed  illustrations.  The  construction  of  the  cutout  figures  by  the  experimenters 
may  have  provided  a  type  of  organization  not 
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present  whc»n  thi»  children  rmiHt rurtod  t hi'ir  own  rcprcHOhtntionH.  Those  results  suggest  a 
possible  connection  between  experimenter-provided  organization  of  visual**  and  increased 
cognitive  processing, 

Levin,  Anglin  and  Carney  (IHM7)  conducted  a  meta-analysis  of  these  studies 
involving  prom*  learning  and  vihujiIh.  HiihimI  on  thin  review,  recommendations  were 
formulated  concerning  the  line  of  vitamin  within  prrisu  instruction,  One  principle  presented 
includes  that  the  function  of  vihuuIh  fir  pictures  should  he  based  on  the  needs  of  the  learner, 
the  instructional  objectives  and  materials  used.  Kuibur  (1994)  interprets  this  principle 
suggesting  that  the       the  function  of  any  visual  must  be  consistent  with  its  intent" 
(p.  141).  Itocnm  inundations  such  oh  these  point  toward  the  essential  role  of  visuals  within 
instruction  and  that  caution  muHt  ho  taken  that  they  not  detract  from  the  content  and  serve 
a  functional  purpose  for  learning, 

Visuals  and  Computer-Based  Instruction 

In  contrast  to  studies  conducted  with  visuals  and  text,  recent  investigations 
involving  visuals  within  computer-based  instruction  is  a  fairly  unexplored  area.  In  response, 
Park  (1991)  has  called  for  an  increased  research  effort  concerning  the  appropriate  selection 
of  information-representation  forms  for  instructional  computing.  Existing  research  has 
included  comparisons  involving  visual  presentation  modes  (Calvert,  Watson,  Brinkley  & 
Penny,  1990),  static  vs.  dynamic  display  of  graphical  information  (King,  1975)  and  the  use 
of  animation  within  instruction  (Reiber,  1990).  The  results  of  these  studies  were  mixed  and 
involved  various  conditions  of  presentation  mode,  levels  of  learning  and  findings.  McCuiston 
(1991)  found  that  the  use  of  dynamic  visuals  within  a  college-level  computer-assisted  lesson 
on  descriptive  geometry  concepts  facilitated  spatial  ability  skills,  but  did  not  increase 
content  acquisition.  The  college  students  scoring  in  the  lower  25%  of  the  content  acquisition 
test  who  viewed  static  visuals  achieved  higher  scores  than  those  who  viewed  a  dynamic 
presentation.  These  conclusions,  however,  are  based  on  computer  manipulation  of  visuals 
and  cannot  he  extended  to  the  learner  manipulation  of  visuals. 

Visuals  and  Cueing  Strategies 

Dwyer  (1978)  has  also  substantially  added  to  the  literature  by  investigating  the  use 
of  visuals  and  expository  text  within  instruction.  He  maintains  that  for  optimal  learning  to 
occur  with  visualized  instruction,  the  learner  must  be  able  to  "... locate,  attend  to,  and 
interact  with  the  relevant  instructional  stimuli  while  ignoring  or  minimizing  the  effect  of  the 
competing  irrelevant  stimuli"  (Dwyer,  1978,  p.  156).  Cautioning  against  the  use  of  complex 
or  realistic  of  visuals  within  instruction,  Dwyer  advocates  the  use  of  cueing  to  highlight  the 
essential  information  from  other  types  of  stimuli.  Reviews  of  these  studies  conclude  that 
visual  presentation  of  information  should  attract  and  sustain  the  learners  attention  without 
adding  unnecessary  attention-gaining  devices  that  may  impede  learning.  These  conclusions 
suggest  that  involving  the  learner  with  the  visual  (i.e.,  by  selecting  and  moving  it)  may 
optimize  learning. 

The  use  of  visual  cueing  strategies  has  also  been  shown  to  be  effective  within 
computer-based  instruction.  Canelos,  Dwyer,  Taylor,  Belland,  and  Baker  (1989)  found  the 
use  of  imagery  cues  and  attention  directing  strategies  to  be  effective  with  adult  learners. 
Presentation  modes  and  cueing,  as  well  as  the  use  of  partial  pictures  and  system  provided 
organization  of  visuals  have  been  shown  to  be  important  instructional  strategy  variables  to 
be  considered  in  the  design  of  visual  instruction. 

Generative  Learning  Theory 

The  generative  process  of  learning  holds  that  it  is  necessary  for  the  learner  to  have 
an  active,  participatory  role  within  the  instruction  in  order  to  construct  and  interpret 
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individual  meaning  of  information.  Wittrock's  (1974a,  b)  construct  of  generative  learning 
defines  the  purpose  of  learning  as  the  generation  of  abstract  and  distinctive,  concrete 
associations  between  prior  experience  and  incoming  stimuli.  Research  in  this  area  has 
primarily  been  carried  out  with  textual  information  such  as  word  lists,  word  pairs, 
sentences,  headings,  word  problems  (e.g.,  Wittrock,  1990;  Wittrock  &  Carter,  1975),  but  has 
also  included  some  studies  involving  visual  elements. 

Linden  and  Wittrock  (1981)  taught  children  to  generate  interpretations  of 
paragraphs  and  create  images  relating  sentences  to  one  another.  In  addition,  children  were 
taught  to  generate  relations  between  stories  they  read  and  their  own  experience.  With  time 
held  constan*.,  the  results  indicated  about  a  50%  increase  in  comprehension  attributed  to 
the  generative  activities  (Linden  &  Wittrock,  1981).  In  a  related  study  with  5th  grade 
students,  Bull  and  Wittrock  (1973)  found  that  drawing  pictures  of  vocabulary  words 
enhanced  recall  of  definitions  among  the  subjects  one  week  later.  Mode  of  generation  did  not 
seem  to  be  the  primary  factor  in  these  studies.  The  important  variable,  however,  was  the 
encouragement  and  facilitation  of  processing  by  the  learner.  The  learning  environment 
should  facilitate  some  type  of  generation  within  the  learner.  Generating  relationships 
between  the  parts  of  the  information  may  build  stronger  connections  in  the  learner's  mind 
compared  to  simply  reading  the  material. 

Direct  Manipulation  and  Organizational  Strategies 

Generative  activities  can  include  various  types  of  interaction  with  instructional 
content  including  paper  and  pencil  activities,  and  manipulating  concrete  objects  (Bull  & 
Wittrock,  1973;  Bennett,  1991).  Current  computer  technologies  are  also  well-suited  to 
providing  generative  experiences  through  the  direct  manipulation  of  on<screen  visuals. 
Direct  manipulation  refers  to  a  type  of  computer  interface  in  which  learners  use  a  mouse  to 
manipulate  spatially  organized  objects  on  the  screen  (i.e.  click  and  drag)  (Whiteside,  Wixon, 
&  Jones).  This  type  of  interface  may  potentially  contribute  to  learning,  Schneiderman 
(1992)  contends  that  physical,  spatial,  or  visual  representations  of  information  appear  to  be 
easier  to  retain  and  manipulate  than  textual  or  numeric  representations.  Carrol,  Thomas 
and  Malhotra  (1980)  found  that  students  who  manipulated  spatial  representations  were 
faster  and  more  successful  in  problem  solving  than  subjects  given  similar  problems  without 
spatial  representation. 

The  direct  manipulation  of  computer  visuals  by  the  learner  must  incorporate  an 
instructional  purpose  in  order  to  constitute  a  viable  learning  strategy.  The  movement  of 
visual  elements  toward  an  intentional  goal  is  a  strategy  to  involve  the  learner  for  the 
purpose  of  increasing  cognitive  processing.  Learning  strategies  assist  the  learner  in 
selecting,  acquiring,  constructing  and  integrating  new  information  (Weinstein  &  Mayer, 
1986).  Organizational  learning  strategies  include  ordering  or  grouping  items  which  activate 
two  parts  of  the  encoding  process  -construction  and  integration.  Construction  involves  a 
process  that  builds  connections  between  thoughts  and  develops  a  cognitive  organization  for 
them,  while  integration  includes  connecting  new  information  with  prior  knowledge 
(Weinstein  &  Mayer,  1984;  1986).  Learner  manipulation  of  on-screen  elements  into  a  target 
visual  organization  may  support  the  learner  in  the  process  of  connecting  information  about 
the  individual  visual  items  to  other  items  in  long  term  memory.  Jonassen  (1988)  states 
that  organizational  learning  strategies  can  assist  in  the  structuring  and  restructuring  of  the 
learner's  knowledge  base  by  clarifying  how  the  concepts  relate  to  one  another. 

PKOULKM  STATEMENT 

The  purpose  of  this  study  was  to  explore  how  the  active,  spatial  manipulation  of 
visual  elements  by  the  learner  mvi  I he  placement  of  them  into  an  organizational  framework 
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can  assist  in  increased  cognitive  processing  and  achievement.  Specifically,  this  research 
drew  upon  generative  processing  theory  to  investigate  the  effects  of  two  forms  of  visual 
organizational  strategies  (cued  and  uncued)  on  four  types  of  learning  objectives.  It  was 
hypothesized  that  those  who  participated  in  the  cued  and  uncued  organizational  treatments 
would  become  more  engaged  in  the  content  and  would  achieve  significantly  higher  scores  on 
the  posttests  than  the  control  group.   It  was  also  hypothesized  that  those  students  in  the 
uncued  organizational  treatment  group  would  require  a  higher  level  of  attention  and 
cognitive  processing  and  thereby  recall  more  facts  and  concepts  and  solve  more  problems 
than  those  who  were  provided  a  cued  organization. 

METHOD 

Subjects 

This  study  involved  74  volunteer  college  students  enrolled  in  an  undergraduate 
statistics  course  at  a  large  eastern  university.  Individuals  were  randomly  assigned  to  one  of 
the  three  treatment  groups.   No  attempt  was  mnde  to  group  the  subjects  by  ability  or 
reading  level,  however,  previous  knowledge  of  human  physiology  was  assessed  prior  to 
instruction. 

Materials 

The  content  of  the  lesson  used  in  this  study  was  adapted  from  paper-baaed  tt?xt 
materials  (2000  words)  developed  by  Dwyer  and  Lamberski  (1977)  concerning  the 
physiology  and  function  of  the  human  heart.  The  lesson  includes  factual,  conceptual  and 
problem  solving  information  covering  the  parts  of  the  heart,  circulation  of  blood,  cycle  of  blood 
flow  and  blood  pressure.  The  basic  lesson  was  re-created  in  a  computer-based  instruction 
format  programmed  using  Authorware  Professional  for  the  Macintosh.  A  pretest  consisting 
of  36  questions  on  the  basic  physiology  of  the  human  body  was  initially  given  to  the 
students  to  assess  the  effect  of  prior  knowledge  on  posttest  scores. 

Instructional  Treatments 

Three  treatments  were  developed  for  this  study  which  contained  identical 
instructional  content  while  varying  the  levels  of  manipulation  and  visual  organizational 
cueing. 

Learner-Generated  Cued  Visual  Organization.  Subjects  in  the  Cued 
Visual  Organization  group  were  required  to  move  (click  and  drag)  a  visual  graphic  of  a  part 
of  the  heart  from  the  left  side  of  the  screen  onto  a  frame  of  the  whole  heart  containing  tan 
outline  of  the  primary  parts,  This  outline  served  as  an  organizational  guide  for  the  correct 
placement  of  the  visuals  (see  Figure  1).  Subjects  received  instructions  to  place  the  part  in  its 
correct  location  on  the  outline  of  the  heart.  A  pre-lesson  exercise  was  provided  to  familiarize 
subjects  with  mouse  control  and  to  allow  for  practice  with  the  manipulation  and  placement 
of  visuals. 

No  text  was  presented  prior  to  the  movement,  forcing  the  learner  to  first  locate  the 
correct  position  on  the  heart,  After  successful  placement  of  the  part,  textual  information 
describing  the  part,  its  location,  function  and/or  relationship  to  other  parts  of  the  heart 
appeared  next  to  the  visual.  If  the  part  was  not  placed  correctly,  u  would  automatically 
return  to  its  original  position  in  the  left  hand  corner  of  the  screen,  providing  animated 
negative  visual  feedback.  The  subject  was  required  to  place  the  visual  again  until  correct. 


Insert  Figure  1  about  here 
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Learner-Generated  Uncued  Visual  Organization.  The  Learner-Generated 
Uncued  Visual  Organization  treatment  group  utilized  a  similar  strategy.  After  an  identical 
pre-lesson  exercise,  the  Uncued  Organizational  treatment  group  was  required  to  move  the 
parts  of  the  heart  onto  a  frame  of  the  whole  heart  containing  no  outlined  visual 
organizational  guide  concerning  the  parts*  location.  As  the  lesson  progressed,  the  visual  of 
the  heart  was  constructed  part  by  part  encouraging  the  learner  to  actively  build  the  visual 
connections  from  :he  parts,  their  location  and  their  function  within  the  heart  to  the  textual 
information  (see  \  sure  2).  As  in  the  Cued  Visual  Organization  group,  when  not  placed 
correctly,  the  part  would  automatically  return  to  its  original  position  on  the  screen  and  the 
lesson  would  not  advance.  As  with  the  Cued  Organization,  after  successful  placement  of  the 
part,  textual  information  describing  the  part,  its  location,  function  and/or  relationship  to 
other  parts  of  the  heart  appeared  next  to  the  visual. 


Insert  Figure  2  about  here 


Control  (System-Provided  Entire  Visual).  The  control  group  presented  the 
lesson  content  via  text  and  graphics  in  a  page-turning  format  with  no  learner  manipulation 
of  visuals.  Subjects  viewed  graphics  with  specific  parts  of  the  heart  highlighted  and  their 
corresponding  textual  information.  The  only  action  required  by  the  learner  was  proceeding 
to  the  next  frame. 


Dependent  Measures 

The  posttest  was  a  pencil  and  paper  test  consisting  of  a  total  of  90  questions 
measuring  different  types  of  educational  objectives.  The  identification  test  consisted  of 
twenty  multiple-choice  items  requiring  students  to  identify  numbered  parts  on  drawings  of 
the  heart.  The  terminology  test  consisted  of  twenty  multiple-choice  questions  designed  to 
measure  knowledge  of  facts,  terms  and  definitions.  The  comprehension  test  consisted  of 
twenty  items  and  tested  the  ability  of  the  student  to  use  the  information  to  explain  another 
phenomenon  occurring  simultaneously.  The  drawing  test  required  students  to  reproduce  a 
diagram  of  the  heart  and  place  numbers  of  the  twenty  parts  in  their  correct  positions 
measuring  the  student's  ability  to  construct,  and/or  reproduce  items  in  their  appropriate 
context.  Finally,  a  problem  solving  test  of  ten  multiple  choice  questions  tested  the  students' 
ability  to  transfer  understanding  of  the  information  and  to  solve  problems  related  to  the 
content.  The  following  reliability  coefficients  (KR-20)  were  recorded  for  each  of  the  tests:  .93 
total  tost,  .82  Identification,  ,79  terminology,  .80  comprehension,  .83  Drawing,  and  .583 
problem  solving.  A  paper  and  pencil  pretest  was  also  conducted  prior  to  the  instruction. 
Tin?  pretest  consisted  of  36  questions  on  human  physiology. 
Procedures 

The  study  was  conducted  in  one  of  the  University  computer  labs.  As  the  students 
arrived  thry  were  randomly  assigned  to  one  of  three1  treatment  groups.  After  a  brief 
presentation,  the  students  individually  took  the  pretest  followed  by  the  instruction.  Upon 
completion  of  the  instruction,  the  posttests  were  immediately  administered  in  the  following 
succession:  drawing,  identification,  terminology,  comprehension  and  problem-solving  tests. 
Individualized  instruction  was  conducted  via  computer  while  the  pretest  and  final  testing 
were  delivered  in  traditional,  paper  and  pencil  format. 
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Research  Design  and  Data  Analysis 

This  study  encompassed  a  post-test  only  control  group  design  controlling  for  prior 
knowledge.  The  variables  manipulated  were  learner-generated  manipulation  of  visuals  with 
cued  organization  and  learner-generated  manipulation  of  visuals  with  uncued  organization 
and  control  (system-provided  organization). 

Separate  one  way  analysis  of  covariance  (ANCOVA)  was  used  to  test  for  significant 
differences  among  the  three  groups  for  each  criterion  measure:  drawing,  terminology, 
identification,  comprehension,  problem  solving  and  total  test.  A  .05  alpha  level  was  selected 
to  determine  significance  among  the  three  treatments.  Due  to  the  exploratory  nature  of  this 
study,  post  hoc  comparisons  were  conducted  to  analyze  the  differences  between  groups  for  all 
measures. 

RESULTS 

Descriptive  results  for  achievement  showing  the  means  adjusted  for  prior  knowledge 
are  reported  in  Table  1.  Total  score  approached  overall  significance  jj>.05  (F(70,2)=3.10, 
MS=288.74,  MSE=93.15).  Examining  the  subscores  revealed  a  significant  effect  for  problem 
solving  £<.05  (F(70,2)=3.58,  MS=12.49,  MSE=3.49). 

Since  the  means  for  the  cued  visual  group  were  higher  for  each  dependent  measure, 
exploratory  post-hoc  comparisons  were  run  between  groups  for  each.  Based  on  this  data,  an 
interesting  trend  was  discovered.  For  the  identification,  problem  solving  and  drawing  tests, 
tluk  cued  visual  treatment  scored  significantly  higher  than  the  uncued  visual  treatment,  but 
not  signify  uiit ly  higher  than  the  control  group.  For  problem  solving,  the  control  group  also 
mpiiiviI  significantly  higher  than  the  uncued  group.  The  remaining  two  tests,  comprehension 
mill  terminology  were  not  significantly  different  at  £=,068,  063,  respectively. 


DISCUSSION 

Thin  exploratory  study  investigated  the  relationship  of  the  direct  manipulation  of 
vimuil  element*  toward  an  organizational  framework,  providing  varying  levels  of  cued 
information  and  achievement  on  four  types  of  learning  objectives.  The  results  suggested 
some  nitei  esting  conclusions.  Of  primary  importance  was  that  fact  that  in  no  case  were  the 
scores  on  either  treatment  group  involving  manipulation  of  visuals  significantly  greater  than 
the  control  group  which  provided  the  visuals.  These  results,  although  theoretically 
somewhat  unfounded,  support  the  findings  of  Lesgold,  Levin,  Shimron  and  Guttman  (1975). 

Although  it  was  expected  that  the  manipulation  of  visual  elements  would  he  more 
effective,  especially  when  learners  created  their  own  organization,  this  hypothesis  was  not 
supported.  Learner- generated  uncued  visual  organization  was  perhaps  too  cognitively- 
demanding  thus  exerting  a  detrimental  effect  on  achievement.   Given  no  visual  structure, 
the  students  expressed  difficulty  inferring  the  locations  of  the  parts  of  the  heart.  The 
majority  of  their  cognitive  processing,  therefore,  may  have  been  utilized  by  unstructured 
guessing,  leaving  very  little  attention  available  to  associate  textual  information  with  the 
visual. 

The  second  interesting  and  important  finding  of  this  study,  supports  the  inclusion  of 
visual  cues  when  incorporating  a  generative  learning  strategy.  In  post  hoc  comparisons  for 
three  of  the  dependent  measures  (Identification,  Problem  Solving  and  Drawing),  the  cued 
visual  organization  group  scored  significantly  higher  than  the  uncued  visual  organizational 
group.  These  results  lead  to  some  interesting  speculation,  When  little  or  no  information  is 
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provided  in  a  generative  strategy,  learners  may  become  frustrated  and  inattentive  (reverting 
to  trial  and  error  guessing)  when  required  to  make  a  specific  right  or  wrong  response.  The 
learner  cannot  construct  his  or  her  own  organization  since  a  specified  one  is  already 
expected.  The  generative  nature  rests  solely  in  the  movement,  but  not  the  placement  of  the 
visual.  With  the  system  cues  present,  locating  the  visual  may  have  reduced  frustration  and 
enabled  the  learner  to  concentrate  on  the  part's  label  and  location. 

Another  speculation  regarding  these  findings  is  related  to  the  three  types  of 
dependent  measures  which  were  significant.  The  generative  activity  required  the  learner  to 
move  visuals  which  were  labeled  onto  a  framework,  closely  representing  the  cognitive 
processing  required  on  the  tests.  The  most  interesting  finding  was  the  impact  of  this 
activity  on  problem  solving.  Upon  closer  examination  of  the  problem  solving  questions,  it 
became  quite  evident  that  without  being  able  to  identify  the  parts  mentioned  (i.e.,  right 
atrium,  tricuspid  valve,  vena  cava,  etc.),  the  learner  would  not  be  able  to  understand  the 
question.  While  not  directly  supporting  the  notion  that  generative  learning  or  cueing 
enhances  problem  solving,  it  does  support  the  learning  of  prerequisite  knowledge  necessary 
for  problem  solving. 

While  the  generative  nature  of  manipulating  the  visuals  showed  no  significant  main 
effect,  the  combination  of  this  type  of  activity  and  a  cued  organizational  framework 
displayed  an  interesting  trend.  This  strategy  supports  strengthening  the  organizational 
cues  when  combined  with  the  manipulation  of  visuals.  In  each  case,  means  were  higher  for 
the  cued  organization,  although  not  significantly,  than  the  control.  The  combined  strategy 
points  toward  a  potentially  integrated  generative  strategy  through  which  the  learner 
interacts  with  sensory  information.  Manipulating  the  part  in  addition  to  targeting  a  visual 
structure  may  provide  a  cueing  strategy  and  assist  the  luurner  in  cognit  ivoly  organizing  the 
information. 

This  study,  while  exploratory  in  nature,  provided  valuable  information  in  regard  to 
the  generative  manipulation  of  visual  elements  into  an  organizational  framework.  Distinct 
from  traditional  system-provided  animation,  this  strategy  capitalized  on  the  direct 
manipulation  interface  capability  of  authoring  software.  Yet  researchers  and  designers 
must  consider  the  effects  of  this  strategy  on  learning  and  further  research  questions  remain. 
How  much  visual  organizational  cueing  is  necessary  in  combination  with  manipulation  to 
encourage  optimal  cognitive  processing  by  the  learner?  Should  textual  information  be  given 
prior  to  the  movement  of  the  visual  so  that  learners  are  provided  with  more  information  with 
which  to  organize  the  visuals?  Shdiild  the  learner  create  his  or  her  own  organization  without 
a  framework  or  outline,  based  on  previously  presented  textual  information?  What  other 
factor  may  be  potentially  masking  the  generative  effect?  The  results  and  positive  trends 
found  in  this  study  warrant  further  research  in  the  area  of  learner-generated  manipulation 
of  visuals  and  cued  organizational  frameworks. 
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Table  and  Figure  CaptioiiH 
Figure  1.  Learner-Generated  Cued  Visual  Oruaiu/.af  ion  (HKK  APIM4NDIX  V) 
Figure  2.  Learner-Generated  Uncued  Vimial  Organization  (HKK  APPKNDIX  V) 
Table  1.  Adjusted  Means  and  Standard  Deviation** 
Table  2.  Analysis  of  Covariance 
Table  3.  Significance  Levels  Post-Hoc  Comparisons 

Table  1 

Adjusted  Means  and  Standard  Deviations 


System-Generated  Learner-Generated 
Organization  Organization 


Cued 

Uncued 

Identification 

12.98 

14.11 

12.49 

Terminology 

11.99 

12.79 

10.87 

Comprehension 

10.78 

12.34 

10.44 

Problem-Solving 

4.77 

4.72 

3.53 

Drawing 

13.54 

14.56 

12.07 

Total  Score 

40.53 

43.95 

37.32 

Covariate  is  pretest  score 


Table  2 
Analysis  of  Covariance 




DF 

Sum  of 
Squares 

Mean 
Square 

F  Value 

Significant*! 
of  V 

Identification 

2 

35.86 

17.93 

2.69 

.075 

Terminology 

2 

48.63 

24.31 

1.73 

.185 

Comprehension 

2 

52.42 

26.21 

1.96 

.148 

Problem-Solving 

2 

24.98 

12.49 

3.58 

.033** 

Drawing 

2 

80.93 

40.47 

2.58 

.083 

Total  Score 

•  2 

577.48 

288.74 

3.10 

.051 

Covariate  is  pretest  score 
**  Significant  at  the  .05  level 


Table  3 
Significance  Levels 
Post-Hoc  Comparisons 


Control 
vs. 

Cued  Organization 

Cued  Organization 
vs. 

Uncued  Organization 

Control 
vs. 

UncuedOrganization 

Identification 

.151 

.026** 

.518 

Terminology 

.480 

.068 

.318 

Comprehension 

.159 

,063 

.754 

Problem-Solving 

.921 

.024** 

.028** 

Drawing 

.395 

.027** 

.218 

Total  Score 

.240 

.015** 

.267 
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Notes 

1.  Legend 

Control  =  System-Provided  Organization 

Cued  Organizations  Learner-Generated  Cued  Visual  Organization 
Uncued  Organization  =  Learner-Generated  Uncued  Visual  Organization 

2.  Cuvariate  is  pretest  score 

U.  **  Significant  at  the  .05  level 

4.  Nut  all  analyses  of  variance  are  significant  at  the  .05  level. 
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Abstract 


The  concept  of  information  landscapes  has  been  a  constant  theme  in 
the  development  of  interactive  multimedia  packages.  For  the  user 
interface  to  this  information  to  be  effective  and  efficient 
consideration  must  be  given  to  the  cognitive  load  placed  on  the  user. 
Improvement  in  learning  outcomes  can  be  supported  by  allowing 
students  to  focus  on  metacognitive  processes  as  a  component  of 
performance  support.  The  renewed  interest  in  student-centred 
learning  environments  and  recent  constructivist  approaches  to 
learning  place  responsibility  for  learning  firmly  on  the  shoulders  of 
the  student,  but  this  responsibility  can  only  be  taken  on  when 
appropriate  support  is  available  and  the  necessary  skills  can  be 
developed  with  students.  This  paper  examines  these  issues  in  the 
context  of  the  development  of  a  CD-ROM  based  interactive 
multimedia  package,  'Investigating  Lake  Iluka\  and  reviews 
evaluation  of  the  learning  outcomes  from  initial  use  of  the  package. 


Over  the  last  decade  there  has  been  a  significant  shift  in  emphasis  in  curricula  generally. 
Learning  basic  facts  and  definitions  from  textbooks  has  become  less  important  than  the 
application  of  knowledge  in  daily  life  and  the  development  of  higher  order  thinking  skills 
such  as  problem-solving,  critical  thinking  and  decision  making.  In  many  countries  this  shift 
has  been  developing  in  parallel  with  national  programs  which  are  emphasising  a  mo\e 
toward  a  more  literate  populus.  The  quality  of  the  learning  outcomes  of  a  nation's  education 
and  training  systems  play  a  central  role  in  determining  the  future  levels  of  economic  and 
social  development.  Many  in-house  industry  training  programs  have  realised  that  the  focus 
should  be  on  effective  performance  and  problem  solving  rather  the  ability  to  remember  facts 
and  repeat  theory  without  real  understanding  about  its  applicability 

Recent  curriculum  documents  in  many  western  countries  emphasise  the  skills  of 
investigation,  reflection  and  analysis  to  generate  or  refine  knowledge.  The  appeal  of 
cognitive  process  training  to  support  this  development  is  obvious,  and  it  seems  far  more 
efficient  to  provide  the  student  with  general-purpose  problem  solving  than  instruction  on 
specific  solutions  to  specific  problems.  Metacognitive  support  provides  a  key  to  efficient 
higher  order  learning. 

There  has  been  considerable  controversy,  which  will  no  doubt  continue  to  simmer,  over  the 
clarification  of  constructivism  as  opposed  to  subjectivism.  The  constructivists  argue  that 
learning  outcomes  depend  on: 

•the  learning  environment. 

•the  prior  knowledge  of  the  learner. 

•the  learner's  view  of  the  purpose  of  the  task. 

•the  motivation  of  the  learner. 

The  process  of  learning  involves  the  construction  of  meanings  by  the  learner  from  what  is 
said  or  demonstrated  or  experienced.  The  role  of  the  teacher  is  one  of  facilitating  the 
development  of  understanding  by  selecting  appropriate  experiences  and  then  allowing 
students  to  reflect  on  these  experiences. 

To  the  learner,  the  constructivist  learning  experience  may  not  look  welcoming.  It  may  seem 
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daunting  and  complex  to  those  who  feel  ill-prepared  for  such  creative  freedom  and  choice  of 
direction.  Often  constructivist  learning  situations  suddenly  throw  students  on  their  own 
management  resources  and  many  fend  poorly  in  the  high  cognitive  complexity  of  the  learning 
environment.  Cognitive  support  tools  and  the  explicit  acknowledgment  of  the  double  agenda 
of  metacognitive  self- management  and  learning  can  help.  The  scaffolding  and  coaching  of  the 
cognitive  apprenticeship  model  offer  another  solution. 


Multimedia  Design  in  a  Contructivist  Framework 

A  number  of  multimedia  design  models  have  been  developed  which  illustrate  th«j 
combination  of  complex  learning  environments  and  which  also  give  students  their  own  real 
control  over  their  learning  environment.  Our  model  (Figure  1)  is  based  on  a  more  organic  mid 
iterative  approach  than  traditional  instructional  systems  design.  Phase  one  takes  the  basic 
information  derived  from  a  needs  assessment  and  converts  it  into  a  description  of  the  Project 
space — the  information  which  is  to  be  included  in  the  materials,  how  it  is  structured,  what 
the  target  audience  understands  about  the  information  and  how  it  might  be  structured  for 
the  audience.  A  possible  structuring  device  might  a  concept  map  of  the  ideas  and  links  that 
are  to  be  included  in  the  project. 

The  second  phase  reviews  the  basic  description  and  seeks  to  link  the  elements  through  an 
appropriate  instructional  or  presentation  strategy.  It  also  seeks  to  identify  metaphors  which 
help  both  the  design  team  and  the  final  presentation  of  the  information  structure.  The 
outcome  of  the  second  phase  would  be  a  formal  description  such  as  a  design  brief.  The  detail 
would  enable  the  reader  to  understand  the  underlying  knowledge  structures  and  the  ways  it 
is  proposed  to  link  them  conceptually  and  intuitively. 


r.  j  g 

249 

9 

ERIC 


User/Designer 


I  View 

Complete  Design  Brief  f 


Figure  1:  The  design  process  used  for  this  interactive  multimedia  package. 

The  third  phase  is  a  third  pass  at  the  same  material,  this  time  with  the  express  goal  of 
linking  the  design  ideas  into  a  potential  interaction  structure.  One  output  of  this  phase 
would  be  an  interactive  mock-up  of  the  interactive  materials  using  such  tools  as  HyperCard 
or  Toolbook  to  illustrate  not  only  static  display  of  informatics  but  also  the  graphical  and 
visual  metaphors  used  to  create  understandable  links.  The  information  included  in  this 
prototype  may  include  visual,  motion,  static  graphics,  sound  and  data  landscapes  as 
appropriate  to  the  concept  under  development. 

Each  interaction  consists  of  a  node  point  which  forms  the  basis  of  the  interaction,  a  set  of 
options  which  provide  links  to  other  nodes  or  additional  information  attached  to  the  current 
node.  One  of  the  links  must  relate  to  earlier  travelled  or  preferred  paths  through  the 
materials,  and  each  choice  must  inform  the  user  about  what  is  likely  to  occur  as  a  result  of  a 
choice.  These  can  translate  into  the  traditional  concept  of  results  (correct  or  incorrect)  or 
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information  feedback  choice,  but  should  also  include  simple  feedback  elements  such  as 
confirmation  of  choice  (feedback  that  a  button  has  been  selected)  or  performance  support 
enhancement  such  as  suggested  hints,  or  revision  of  the  underlying  concept/principle  which 
might  be  employed  to  make  the  choice.  Depending  on  the  instructional  strategy  chosen 
another  element  might  include  the  concept  of  duration,  either  time  or  the  limit  of  options 
based  upon  previous  choices  or  paths  taken.  What  constitutes  each  of  these  functions  and 
what  they  create  in  terms  of  cognitive  skill  development  for  the  user  are  determined  by  their 
physical  manifestation  in  terms  of  navigation  options. 

The  complex  integration  now  possible  with  a  variety  of  hardware  and  software  combinations 
raises  problems  for  the  user  in  tL  *  multiple  paths  are  possible  to  the  same  or  different  end- 
points.  Learners  are  faced  with  the  need  to  understand  what  learning  possibilities  might  be 
available  from  where  they  are  in  a  multimedia  learning  environment.  When  a  student  can 
branch  down  multiple  paths  and  rapidly  change  the  direction  and  focus  of  the  learning 
sequence,  there  is  possible  interference  with  effective  learning  through  the  inappropriate 
application  of  information  by  the  learner  to  their  internal  schemas.  Other  concerns  include 
disorientation  with  location,  cognitive  overload  when  following  several  trails  or  trying  to 
remain  oriented  on  their  goal,  flagging  commitment  and  a  poor  presentation  rhetoric  or 
metaphor. 

The  potential  of  both  the  technology  and  learning  strategies  to  incorporate  the  recent 
initiatives  in  science  education  have  lead  to  the  development,  production  and  evaluation  of  a 
particular  interactive  multimedia  CD-ROM  based  package  called  Investigating  Lake  Iluka. 
The  package  has  been  designed  to  facilitate  access  to  the  information  landscape  through  the 
learner's  choices  by: 

*  supplying  accessible  and  useable  tools  to  allow  access  to  the  scope  of  supporting 
interactive  multimedia  resources  (  eg.  video  and  graphic  representations  of  concepts) 

•  providing  an  adaptive  navigation  system  and  coherent  information  metaphor  which 
requires  little  or  no  explanation. 

Investigating  Lake  Iluka  has  been  based  around  an  ecology  simulation  and  employs  a 
number  of  different  interface  metaphors  in  presenting  the  materials  to  the  user.  The  package 
is  based  on  the  concept  of  an  information  landscape  that  incorporates  the  biological,  chemical 
and  physical  components  of  a  range  of  ecosystems  that  make  up  a  coastal  lake  environment, 
The  user  is  given  some  problem  solving  strategies  to  investigate  thia  information  in  a  variety 
of  ways  using  the  range  of  physical  tools  provided.  They  can  collect  biological,  physical  or 
chemical  data  as  well  as  media  information  and  'construct'  t  heir  own  understanding  of  the 
basic  ecology  concepts  embedded  in  the  package.  This  facility  has  the  potential  to  increase 
student  understanding  and  control  of  their  learning  through  control  of  their  learning 
environment.  Inquiry  and  problem-solving  techniques  have  been  embedded  in  the  package 
through  case  studies  of  ecological  scenarios  presented  to  the  user  via  media  reports  of 
problems  posed  directly  to  the  user.  Each  scenario  can  he  investigated  using  the  package 
tools.  It  is  expected  that  users  will  develop  a  broad  array  of  scientific  investigation  skills 
using  this  realistic  simulation.  One  of  the  unique  features  of  this  package  is  the  facility  for 
users  to  generate  their  own  customised  report  which  can  be  refined  and  presented 
independently  of  the  package. 

Inquiry  and  problem-solving  techniques  have  been  embedded  in  the  package  through  case 
studies  of  ecological  scenarios  presented  to  the  user  via  media  reports  of  problems  posed 
directly  to  the  user.  Each  scenario  can  be  investigated  using  the  package  tools.  It  is  expected 
that  users  will  develop  a  broad  array  of  scientific  investigation  skills  using  this  realistic 
simulation.  One  of  the  unique  features  of  this  package  is  the  facility  for  users  to  generate 
their  own  customised  report  which  can  be  refined  and  presented  independently  of  the 
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Urban 

Problems 


|  Hint*  | 


Media 
Information 

^/  •  1  -  Ho  v  can  urban 
communities  protect  a  laKe 
environment  from  pollution? 


Hint: 

MEDIA  -  bnrwe  the 
nevspapers,  listen  to  tapes 
and  vatch  videos  in  the 
Field  Study  Centre. 
Remember  you  can  use  FIND 
or  the  INDEX  in  the  print 
material.  Decide  vfaat  your 
keyvord  is  and  do  a  search. 


2>    |  Copy  Problem  to^e book  | 
I    More  Problems  I 


Figure  2:  Investigating  Lake  Iluka  where  the  user  is  presented  with  a  problem  and  a  Bet  of 
suggestions  to  help  them  solve  a  problem  not  simply  choose  an  answer  to  prepared  multiple 
choice  questions.  (Program  from  Interactive  Multimedia  Pty  Ltd,  Old  Parliament  Hourse, 
Canberra) 


Metacognitive  Support  in  Investigating  Lake  Iluka 

Paris  &  Winograd  (1990)  have  defined  metacognition  as  knowledge  about  cognitive  states 
and  abilities  that  can  be  shared  among  individuals,  including  the  affective  and  motivational 
aspects  of  thinking.  Cognitive  strategies  can  be  addressed  directly  in  the  structure  of  an 
information  landscape.  The  affective  and  motivational  aspects  of  metacognition  are 
embedded  in  the  interface. 

The  problem  solving  nature  of  Investigating  Lake  Iluka  lends  itself  to  metacognitive  support 
through  a  number  of  means: 

*   cognitive  self-management:  Students  should  form  good  plans,  use  a  variety  of  strategies 
and  monitor  and  revise  ongoing  performance.  The  notebook  within  Investigating  Lake 
Iluka  allows  the  student  to  collect  and  manage  information  from  a  variety  of  sources. 
Transcripts  are  not  provided  for  video  and  audio  material  as  this  is  not  consistent  with 
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how  students  would  process  material  in  these  media  outside  the  package.  They  must 
organise  and  edit  what  is  potentially  an  overload  of  information,  within  t)  :>  broad 
constraints  of  an  open-ended  problem. 

The  role  of  the  teacher  is  vital  to  the  editing  process  at  this  stage.  Students  need  to  be 
encouraged  to  be  critical  in  appraising  the  relevance  and  credibility  of  material.  They 
also  need  to  be  guided  through  the  development  of  a  report  on  the  problem  posed. 

*  provision  of  prompts:  Hints  accompany  the  problems  posedwithin  each  ecosystem. 
Students  choose  to  access  these  hints,  which  guide  them  to  explore  various  areas  of  the 
landscape  or  measure  certain  physical,  chemical  or  biological  characteristics  with  the 
toolkits  provided. 

The  role  of  the  teacher  is  to  ensure  students  record  data  collected  in  a  systematic  and 
scientific  way. 

•  experience  of  experts:  the  reference  book  and  a  number  of  media  reports  give  the  student 
the  chance  to  gain  from  the  experience  of  those  who  specialise  in  ecology. 

The  role  of  the  teacher  is  to  help  students  evaluate  issues  of  hidden  agenda,  conflict 
among  experts,  alternate  sources  of  information  and  timing  of  information  release. 

You  need  specific  and  accessible  knowledge  to  solve  problems.  An  information  landscape  such 
as  Investigating  Lake  Iluka  provides  the  knowledge  base  and  the  knowledge  schemas  of 
experts  in  association  with  a  mechanism  for  the  student  to  collect,  analyse,  assimilate  and 
synthesise  responses  to  problems.  The  ability  to  see  a  bigger  picture  is  facilitated  by  rapid 
information  access  and  retrieval. 

Embedded  content  independent  strategies  are  general  learning  strategies  incoporated  within 
available  content.  They  support  local  learning  but  emphasise  strategy  transfer  as  well 
(Osman  &  Hannafin,  1992) .  A  well  structured  information  landscape  will  provide  a 
template  for  a  range  of  content.  The  strategies  used  in  Investigating  Lake  Iluka  could  well 
apply  to  other  tools  and  other  problems. 

Evaluation  Strategy 

The  evaluation  involved  three  main  approaches.  Expert  review  of  the  package,  one-on-one 
testing  of  the  prototype  materials  via  video  observation  and  interviews  and  in-depth  case 
studies,  including  the  verification  of  the  methods  for  data  analysis  of  complex  multi-path 
data  This  type  of  data  collection  varies  from  subject  to  subject  and  requires  the  development 
of  special  techniques  for  its  analysis  and  interpretation.  Examination  of  the  contents  of  the 
incorporated  notebook  provided  some  indication  of  the  pattern  of  student  use  of  both  physical 
and  metacognitive  tools  in  the  package.  This  formative  aspect  of  the  evaluation  was  be  used 
to  guide  decisions  on  debugging  and  enhancing  the  package.  A  further  evaluation  strategy 
was  employed  with  classroom  groups  of  students,  who  were  set  learning  tasks  individually  in 
the  multi-media  environment,  and  tracking  data  collected  for  analysis. 

Analysis  of  the  collected  data  provides  the  group  with  an  interesting  opportunity  for  the 
continued  development  of  techniques  to  extract  the  maximum  amount  of  information  for 
feedback  to  developers  and  those  who  commission  interactive  materials.  Simple  statistics 
such  as  how  many  users  have  used  the  system  and  the  lengths  of  time  tbry  used  it  are 
relatively  simple  to  extract,  but  more  sophisticated  analyses  of  how  a  particular  interactive 
package  is  actually  being  used  (what  sections  are  being  used,  to  what  extent,  what  is  not 
being  accessed,  where  users  exit  the  system,  by  what  method,  and  so  on)  were  examined  and 
the  results  used  to  provide  feedback  on  the  future  specifications  for  new  versions  and  added 
features. 
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Expert  Review 

A  naive  expert  tester  had  an  educational  background  in  Mathematics  and  Computers.  He 
was  asked  to  review  the  program  and  to  verbalise  his  actions  and  thoughts.  He  had  not  been 
introduced  to  the  program  before.  His  commentary  was  observed  and  recorded,  and 
comments  and  observations  were  added  later. 

The  naive  expert's  responses  were  about  two  main  issues:  interface  conventions  and  the 
conceptual  structure  and  functionality  of  the  package. 

Interface  Conventions 

•  Text  transfer  to  the  Notebook:  he  wanted  to  cut  and  paste.  He  tried  to  "Drag>' across  the 
text  as  a  method  of  'select  all\  and  felt  that  a  click  was  not  a  standard  method  to  cut  and 
paste.  "If  I  click  on  there  (the  text)  I  get  the  whole  the  lot.  I  can't  just  pull  a  sentence  off*. 
He  found  the  process  of  editing  a  little  confusing,  but  realised  this  was  necessary 
otherwise  text  would  be  corrupted  by  tho  novice  user. 

•  The  recording  of  measurements  in  the  Notebook:  he  was  confused  as  to  what  the  pencil  was 

used  for.  lie  felt  once  the  hand  symbol  moved  into  his  notes  it  should  automatically  turn 
into  an  "I"  beam  so  he  could  write.  The  naive  Expert  wanted  to  use  the  pencil  to  activate 
this,  "I  was  looking  for  a  tool  to  turn  the  hand  curser  to  an  "I"  bar. 

Conceptual  Structure  and  Functionality 

Early  in  the  testing  process,  he  realised  that  he  needed  the  help  screens  to  fully  understand 
how  to  use  the  program.  After  the  introduction  finished  he  worked  through  all  of  the  Help 
screens.  "I  now  feel  that  I  have  an  understanding  of  that". 

Within  an  ecosystem,  the  naive  Expert  selected  a  problem  and  used  the  Notebook  to  review 
the  question  and  hints.  He  kept  losing  the  question  which  was  not  at  the  top  of  the  notebook  ( 
he  had  other  data  in  there),  as  the  scroll  bar  jumped  back  to  the  top  each  time  the  notebook 
was  opened  and  closed.  This  was  subsequently  rectified  by  putting  the  heading  "PROBLEM" 
and  a  blank  line  before  the  problem  in  the  notebook,  so  it  was  more  easily  located. 

He  found  measurement  from  the  image  of  the  ecosystem  a  wonderful  idea,  and  set  himself  a 
problem  of  investigating  wind  speed  in  different  areas.  It  took  some  trial  and  error  getting 
measurements  into  the  Notebook  as  the  pencil  was  turned  off  and  also  hidden  by  toolkit.  The 
naive  Expert  expressed  the  desire  to  establish  a  transect  for  measurements  to  make  a  set  of 
multiple  readings. 

Within  the  media  room,  the  naive  Expert  noted  the  video  titles  didn't  give  an  idea  of  content 
and  expressed  a  desire  to  see  these  indexed.  He  opened  the  notebook  while  a  video  was 
playing  and  began  typing.  "That  is  really  good,  that  you  can  play  the  video  and  enter  data/' 

Having  used  the  program  once,  the  naive  Expert  felt  confident  about  its  use  and  moved 
through  the  media  room  easily  on  his  second  visit  while  solving  a  different  scenario  problem. 
He  was  still  confused  about  the  pencil,  but  with  trial  and  error  was  able  to  record 
measurements. 

The  naive  Expert  was  impressed  with  the  full  screen  presentation  of  downloaded  notebook 
contents  for  editing  in  a  word  processor.  The  main  outcome  of  this  evaluation  can  be 
summarized  as  a  series  of  guides  to  faciltators: 

•  Work  through  the  Help  screens  early  in  the  introduction  of  the  program  to  students. 
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•  The  Teacher's  manual  should  emphasise  that  editing  capacity  is  only  on  the  blue  pages  of 

the  Notebook  and  is  activated  by  clicking  within  the  notes. 

•  There  should  be  a  demonstration  early  in  the  presentation  of  the  program  so  students 

know  to  turn  the  pencil  on  to  begin  recording  measurements. 

User  Evaluation  (Teachers  and  Students) 

Evaluation  of  the  package  by  teachers  and  students  in  the  early  stages  of  completion  resulted 
in  a  number  of  insights  into  the  design  features  of  the  application  and  also  helped  to  focus 
the  package  objectives.  The  package  was  introduced  to  two  classes  of  year  11  students 
studying  the  topic  of  ecology.  Extensive  observations  were  made  of  the  student  use  of  the 
package,  as  well  as  their  response  to  the  percieved  outcomes  of  its  use. 

The  information  landscape  structure  of  the  program  relies  heavily  on  teachers  understanding 
of  the  processes  that  can  be  practised  and  the  nature  of  the  problems  posed  in  the  problem 
section  of  the  the  program.  The  performance  support  tools,  such  as  the  notebook,  are  an 
integral  part  of  the  investigative  denign,  but  they  do  rely  on  good  direction  by  teachers.  As 
an  example  of  this  issue  one  of  the  inbuilt  problems  in  the  Mangrove  Ecosystem  was  posed  by 
one  of  the  teachers  in  these  initial  trials— "What  types  of  animals  are  adapted  to  this 
environment?" 


After  setting  up  small  groups,  demonstration  of  the  navigation  process  and  a  brief  practice 
with  the  use  of  the  note  book,  the  class  was  given  the  opportunity  to  use  the  package  over  a 
number  of  class  sessions.  The  expectation  from  the  teacher  was  that  students  would  read  or 
collate  the  relevant  information  from  the  package.  They  would  analyse  the  information  and 
present  a  synthesised  statement  of  the  key  characteristics  of  the  animals  of  the  mangrove 
ecosystem.  The  students  had  no  difficulty  collecting  the  information  via  the  notebook  cut  and 
paste  facility,  but  for  the  majority  of  groups,  the  teacher  recieved  a  complete  printout  of  the 
"Animal  and  Plants  Book"  information  for  every  animal  that  lived  in  the  mangroves.  Thus 
they  have  not  attempted  to  analyse  the  pertinent  information  and  remove  sections  which  did 
not  answer  the  question.  Two  groups  did  attempt  some  synthesis  of  the  information,  but 
again  tended  to  include  rather  than  exclude  irrelevant  information. 

Interviews  with  the  students  and  teachers  revealed  that  they  did  have  skiDs  to  synthesize 
such  data,  and  had  previously  demonstrated  these  skills,  but  the  students  admitted  that  the 
power  of  being  able  to  extract,  in  electronic  form,  every  bit  of  information  on  individual 
animals  from  the  package  compared  with  the  usual  practice  for  such  tasks  of  having  to  type 
in  their  response,  or  write  it  out  by  hand  proved  to  be  too  attractive  not  to  include  all  that 
could  be  collected.  Even  after  a  class  discussion  of  the  type  of  report  the  teacher  was 
expecting  the  importance  of  synthesising  information  to  produce  a  concise  answer,  the  final 
reports  by  students  were  much  longer  than  the  teacher  would  normally  received,  if  written 
resources  were  supplied  to  the  students.  After  using  the  complete  package,  the  skills 
required  together  with  the  necessity  to  edit  and  tighten  reports  was  seen  as  an  important 
learning  point  for  students.  It  was  seen  as  a  new  skill  which  up  to  the  a  vailablity  of  this 
technology  had  not  been  an  issue. 

Observation  of  the  use  of  the  navigation  facilities  indicated  that  the  students  had  little 
difficulty  in  finding  information  in  the  package  that  they  sought.  The  "help"  facilities  were 
only  used  occasionally.  It  is  important  to  note  that  all  the  help  was  provided  in  three  ways. 
Help  on  how  the  package  worked  and  how  to  use  the  measuring  tools  was  provided  by 
animated  movies  of  the  screens  with  accompaning  explanation.  Help  on  how  the  package 
was  organised  conceptually  was  provided  through  a  stack  map  which  was  allways  available 
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and  also  allow  navigational  "jumping"  between  sections,  the  third  form  of  help  was  provided 
through  the  use  of  hints  and  suggestions  about  where  to  look  to  solutions  and  what  might  be 
important  concepts,  this  last  form  was  highly  context  specific  and  provided  specific  ideas 
related  to  the  learning  task. 

Other  Teacher  Evaluations 

Information  was  collected  on  teachers  response  to  the  use  of  the  package  with  their  class  and 
also  through  use  of  the  package  in  workshops  at  local  conferences.  A  number  of  key  issues 
arose  from  this  trialling.  Some  of  the  recommendations  were  incorporated  in  the  release 
version,  and  others  will  he  incorporated  in  future  developments  of  the  package.  Most  of  the 
key  issues  raised  by  teachers  were  not.  to  do  with  the  internal  workings  of  the  package,  but  to 
do  with  the  teaching  process  and  organisation  of  their  classes. 

Many  schools  do  not  have  large  numbers  of  machines  with  CD-ROM  facilities.  Teachers 
were  concerned  that,  even  though  t  hey  believed  that  the  package  would  strongly  support  the 
teaching  process,  lack  of  technology  would  reduce  the  impact,  and  in  some  schools  mean  that 
teachers  would  not  use  the  package.  A  strategy  to  maximise  the  availability  of  the  program 
to  students  was  adopted.  Two  versions  of  the  program  were  placed  on  the  CD-ROM,  one 
which  relied  on  the  video  segments  on  the  CD-ROM  disc,  and  one  which  did  not  use  the  video 
materials  and  could  be  copied  to  hard  disc.  Each  package  then  represented  a  licence  to  use 
twenty  copies  of  the  hard  disc  version  and  one  copy  of  the  CD-ROM  with  the  video  segments. 

The  other  key  issue  raised  by  teachers  was  related  to  appropriate  teaching  and  learning 
strategies.  Many  teachers  wanted  guidance  on  how  to  organise  cooperative  group  learning, 
independent  research  and  guided  inquiry  lessons.  It  was  considered  that  this  teacher  support 
could  not  be  incorporated  into  the  package  directly,  hut  was  added  as  part  of  the  printed 
materials  distributed  with  the  package.  There  was  also  considerable  interest  expressed  for 
ideas  on  how  to  integrate  field  trips  into  use  of  the  package.   Ideas  on  use  of  the  package 
before  visiting  a  field  site  and  on  following  up  the  field  trip  were  considered  to  be  important 
to  add  to  the  support  materials. 

The  teachers  involved  in  the  initial  class  testing  of  the  program  were  particularly  excited 
about  their  students  use  of  the  notebook  concept  and  were  surprised  at  the  ease  with  which 
their  students  used  this  inbuilt  tool.  However,  they  did  find  that  students  took  notes  and 
saved  their  notes  effectively  at  the  end  of  each  session,  but  had  difficulty  determining  what 
data  they  had  previously  collected  when  they  started  a  new  session.  Teachers  proposed  that 
previously  saved  notes  should  be  able  to  be  read  back  into  the  package  so  that  students  could 
more  easily  continue  their  investigation.  This  feature  has  not  been  incorporated  into  the 
current  version. 

There  was  also  a  general  consensus  amongst  the  teachers  that  the  package  had  a  much  wider 
application  than  ecology  education.  Many  teachers  envisaged  application  to  geography  and 
especially  English,  in  the  form  of  media  studies.  One  interesting  aspect  of  the  evaluation 
involved  discussion  of  student  access  to  the  video  and  audio  scripts.  The  design  team  had 
intended  to  incorporate  this  feature,  but  teachers  disagreed.  They  proposed  that  it  was 
necessary  for  students  to  develop  skills  in  summarise  information  from  such  media  sources 
and  if  the  students  had  access  to  a  textual  form,  this  would  negate  development  of  an 
essential  skill  for  students. 

Teachers  evaluating  the  package  were  particularly  supportive  of  a  number  of  features  of  the 
package  that  they  saw  stongly  supporting  the  teaching  and  learning  process.  The  features  of 
the  package  noted  in  this  context  were:- 
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•  the  level  of  interactivity 

•  the  measuring  tools 

•  the  notebook  facility 

•  the  video  and  audio  resources. 

Their  main  concern  was  that  teaching  support  ideas  should  be  available  with  the  package  so 
that  they  could  use  the  package  to  the  maximum  advantage. 

Future  Developments 

On  the  basis  of  this  inital  work  we  have  been  encouraged  that  the  package  does  in  fact 
require  the  user  to  take  control  of  their  learning.  The  metacognitive  supports  do  work  well  to 
provide  a  structure  and  support  for  problem  solving.  The  main  drawback  has  been  the 
reluctance  of  the  students  to  edit  out  redundant  or  unessary  text  in  their  reports.  As  a  result 
it  is  suggested  that  future  versions  of  the  software  incorporate: 

•  Prompts  or  advice  on  report  format  either  though  expert  opinion  or  structure. 

•  Expert  scientists,  how  they  would  approach  the  solution  to  the  problem  (Guides)  (Hints 
mainly  focussed  on  where  to  look  in  the  package.) 

•  Report  generation  incorporating  other  media. 
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TEA3M: 

A  System  for  Infusing  Technology  into  Teacher  Education 


To  meet  the  challenges  of  the  21st  century,  teachers  must  address  the  highly  diverse 
needs  of  today's  students  and  prepare  them  for  life  in  an  information-based,  technology- 
driven  society.  To  respond  to  these  challenges,  the  University  of  Houston-Clear  Lake,  in 
collaboration  with  Houston  area  school  districts  and  communities,  NASA,  and  IBM  have 
established  the  Teacher  Education  Advancing  Academic  Achievement  Model  (TEA^M)  to 
design  and  implement  an  innovative  program  for  teacher  education.  This  paper  addresses 
the  integration  of  technology  and  innovative  instructional  practices  in  the  pre-service  and 
staff  development  training  of  teachers  and  administrators.  More  specifically,  this  paper 
describe  the  systems  approach  taken  to  restructure  and  infuse  technology  into  the  pre- 
service  and  in-service  teacher  education  program  at  the  University  of  Houston-Clear  Lake 
Systems  Approach  to  Infusing  Computer-Technology 

The  high-tech  transformation  of  schools  predicted  by  many  has  yet  to  materialize. 
Schools  today  look  very  much  like  they  did  a  hundred  years  ago.  Compared  to  our 
counterparts  in  business  and  industry,  relatively  little  has  changed  in  practice  or  structure. 
In  few  instances,  one  may  find  a  class  where  students1  huddle  around  computer 
workstations,  actively  engaged  in  group  problem-solving,  working  as  a  team  to  access, 
manipulate,  and  interpret  vast  amounts  of  information  as  an  integral  part  of  their  schooling. 
More  common  is  the  sight  of  students,  herded  through  computer-labs  to  learn  isolated 
computer-skills  or  to  remediate  instruction.  Effective  and  innovative  applications  of 
computer-technology  with  classroom  instruction  remain  isolated. 

A  number  of  factors  contribute  to  the  inadequate  use  of  computer-technology. 
Insufficient  resources,  incongruence  with  touchers'  beliefs,  incompatible  hardware,  top  down 
initiation,  fear  of  change,  lock  of  rewards,  inadequate  software,  and  insufficient  time  or 
training  are  some  of  the  more  well  documented  causes.  Together,  these  factors  suggest  one 
basic  underlying  problem:  failure  to  view  education  ns  a  system,  a  set  of  interrelated 
components  that  must  work  together  to  achieve  a  common  purpose. 

Applications  of  the  "Systems  Approach"  to  education  are  not  new.  Human 
performance  specialists  made  system  concepts  popular  during  the  1960's  and  1970's.  Many 
alternative  models  are  now  available  including  those  of  Senge  (1990),  Banathy  (1987), 
Kaufman  &  Thiagarajan  (1987),  Dick  &  Carey;  (1985),  Doming  (1982),  Branson,  Rayner, 
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Cox,  Furman,  King  &  Hannum  (1976),  Churchman  (1969),  and  Gagne  (1962).  The  purposes 
and  contents  of  each  model  differ  based  on  context  and  the  authors*  interpretation  of 
General  Systems  Theory  (GST).  There  is,  however,  one  basic  concept  common  to  all 
approaches:  A  system  is  a  set  of  interrelated  components  that  work  together  to  achieve  a 
common  purpose.  The  reminder  of  this  paper  describes  how  this  principle  is  being  used  to 
create  a  system  for  infusing  computer-technology  with  teacher  education. 


Defining  System  Goals 

Application  of  the  systems  approach  began  with  goal  definition.  By  first  defining 
their  goal,  system  designers  were  able  to  identify  essential  system  elements  and  focus  their 
resources  without  the  constraints  of  existing  organizational  policies  or  structures.  To  help 
ensure  that  the  goals  for  technology  would  be  aligned  with  other  system  components,  a 
collaborative  mission  definition  process  was  used  with  participation  from  all  system 
stakeholders  (more  information  about  the  mission  definition  process  is  given  later  in  this 
paper). 

The  goal  for  technology  was  defined  as,  "to  ensure  that  all  prospective  and  practicing 
educators  are  technology  proficient.  That  is,  they  will  be  able  to  facilitate  student  and  their 
own  performance  using:  (1)  personal  productivity  fools  (e.g.,  word  processors,  databases, 
spreadsheets,  and  graphic  programs);  telecommunication  tools  (e.g.,  E-mail,  electronic 
networks,  and  on-line  services);  learning  tools  (e.g.,  dhll  &  practice,  tutorials,  simulations, 
multimedia  programs);  learning  management  tools  (e.g.,  gradebooks,  test  banks,  electronic 
portfolio's);  (5)  programming  and  authoring  tools  (e.g.,  HyperCard,  Macromind  Director, 
Linkway,  Asymetics  Toolbook,  Basic,  LOGO,  Pascal),  and  6)  collaborative  tools  (e.g., 
electronic  group  problem-solving,  decision -making,  and  brainstorming  applications).11  It  was 
argued  that  attainment  of  this  goal  was  essential  for  the  collaborative  to  achieve  its 
mission. 

The  second  step  in  designing  the  system  was  to  define  the  interrelated  components 
that  must  work  together  to  achieve  its  goal.  Figure  1  illustrates  the  essential  functional 
components  of  the  system.  This  model  is  based  on  the  operations  model  for  successful 
schooling  developed  by  Florida's  Schoolyear  2000  initiative  (Branson  &  Hirumi,  1994). 
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Figure  1.  TEA^M's  System  for  Infusing  Computer-Technology 
System  Components 

The  system  for  infusing  computer-technology  under  development  by  the  TEA^M 
collaborative  is  made  up  of  nine  interrelated  components.  It  is  believed  that  these  nine 
components  must  be  designed  to  work  together  in  order  to  develop  an  effective  and  efficient 
system  for  infusing  computer  technology  with  teacher  education.  This  system  is  also  viewed 
as  a  subsystem  of  the  larger  teacher  education  system  and  thus,  must  be  designed  to  work 
with  other  components  of  teacher  education.  The  remainder  of  this  section  describes  the 
function  and  design  of  each  system  component. 

Mission  Definition 

The  function  of  mission  definition  is  to:  (1)  define  the  vision  and  mission  for  the 
system;  (2)  define  functional  goals  for  system  components;  (3)  help  define  individual 
objectives;  and  (4)  ensure  that  the  objectives,  goals,  mission,  and  vision  are  aligned  (viz.  the 
achievement  of  individual  objectives  will  result  in  the  achievement  of  functional  goals  that, 
in  turn,  results  in  the  achievement  of  the  organizational  mission,  and  achievement  of  the 
organizational  mission  contributes  to  the  realization  of  the  vision  for  an  ideal  society). 

The  mission  definition  process  for  the  TEA^M  collaborative  was  undertaken  during 
the  planning  stages  of  the  new  program.  Representatives  from  all  system  stakeholders  (i.e., 
family  and  community,  public  schools,  business  and  industry,  and  higher  education)  spent 
an  entire  day  at  UHCL  to  define  the  vision,  mission,  and  functional  goals  for  the 
collaborative.  The  group  stated,  restated,  discussed,  and  argued  until  consensus  was 
reached  on  a  vision  and  mission  for  the  collaborative.  Subcommittee  were  then  formed  to 
define  functional  goals  for  each  component  of  the  TEA^M  collaborative.  Figure  3  depicts  the 
vision  and  mission  for  the  collaborative  and  the  functional  goal  for  technology. 
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Figure  3.  Mission  Definition  for  Computer  Technology 


Research  &  Development 

At  best,  education  has  been  slow  in  translating  technological  advances  and  research 
findings  into  actual  classroom  practices.  While  universities  and  other  research  institutions 
continue  to  conduct  considerable  research  on  education,  technology,  and  learning,  the  results 
of  such  studies  are  rarely  implemented  at  the  school  level.  Some  reasons  for  the  lack  of 
transfer  are  that  research  findings  are  not  easily  accessible  and  are  not  reported  in  a 
manner  that  is  readily  applicable  by  classroom  teachers. 

The  function  of  the  Research  &  Development  is  to  acquire  new  knowledge  related  to 
educational  and  administrative  practices,  and  to  make  this  knowledge  readily  accessible 
and  applicable  for  teachers  and  administrators.  To  help  achieve  these  purposes,  the 
technology  component  of  the  TEA^M  collaborative  is  developing  an  electronic  network  that 
will  link  the  public  schools  directly  to  UHCL  and  to  the  Texas  Education  Network  (TENET) 
and  Internet.  As  a  part  of  the  network,  a  mail  and  file  server  at  UHCL  will  contain  specific 
areas  for  accessing  and  discussing  research  findings  and  classroom  practices  related  to 
education.  Professors  at  UHCL  will  locate  and  synthesize  research  related  various  topics 
(e.g.,  technology,  early  childhood  education,  multicultural  education,  interdisciplinary 
strategies,  etc.)  and  make  this  information  accessible  to  teachers  through  the  server. 

Curriculum 

Curriculum  provides  guidelines  for  what  teachers  should  know  and  be  able  to  do 
with  computer-technology.  It  also  recommends  curriculum  scope  and  sequence.  The  words 
"guidelines"  and  "recommends"  are  used  especially  to  denote  the  flexibility  of  the  program. 
Past  practices  have  shown  that  there  is  no  one  single  set  of  competencies  that  are 
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appropriate  tor  all  teachers,  Nor  is  it  necessary  for  teachers  to  learn  these  many  of  the 
competencies  in  a  specific  sequence.  Relevant  computer  competencies  depend  on  each 
teachers  role  and  responsibility  and  the  context  they  are  to  be  applied.  The  scope  and 
sequence  will  depend,  to  a  great  extent,  on  each  teachers1  interest  and  prior  experience. 

Technology  related  learning  outcomes  for  TEA^M  teachers  are  currently  being 
specified  through  a  collaborative- wide  consensus  process.  The  process  includes  several 
stages.  The  first  stage  was  to  generate  a  comprehensive  list  of  computer-technology  related 
competencies  based  on:  (1)  the  predefined  goal  for  technology;  (2)  a  technology  standard  for 
the  professional  preparation  of  teachers  specified  by  the  collaborative;  and  (3)  a  review  of 
national  and  state  standards,  journal  articles  and  textbooks. 

For  the  second  stage,  currently  in  progress,  focus  groups,  made  up  of  representatives 
from  system  stakeholders  (e.g.,  teachers,  administrators,  district  technology  coordinators, 
university  faculty)  are  being  brought  to  UHCL's  Group  Support  Systems  (GSS)  Lab  to  reach 
consensus  on  essential  computer-technology  related  competencies.  The  GSS  Lab  is  equipped 
with  state-of-the-art  electronic  collaborative  tools.  Here,  fourteen  networkod  workstations 
running  the  software  application,  VisionQuest®,  are  being  used  to  facilitate  brainstorming, 
group  discussion,  and  consensus  building.  During  this  activity,  collaborative  members  arc 
being  asked  to  determine  whether  each  competency  is  either:  (1)  essential  to  all  beginning 
educators;  (2)  essential  to  all  experienced  educators;  (3)  essential  for  only  curtain  educators; 
or  (4)  not  essential. 

After  essential  skills,  knowledge,  and  attitudes  are  defined  for  various  stakeholders, 
they  will  be  grouped  into  logical  clusters  based  on  their  content  and  function.  The  clusters 
will  then  be  used  to  define  learning  outcomes.  It  is  believed  that  specifying  discrete  skills 
and  knowledge  may  inhibit  learners1  ability  to  synthesize  and  apply  the  competencies  in  real 
world  situations.  Thus,  learning  outcomes  will  be  generated  that  describe  how  learners  are 
expected  to  apply  the  learned  competencies.  The  learning  outcomes  will  then  be  used  as  a 
basis  for  instructional  design  and  development. 

Instruction 

The  function  of  instruction  is  to  design,  develop,  and  deliver  instructional  programs 
and  materials  that  will  enable  educators  to  achieve  the  specified  curricular  objectives.  To 
achieve  the  technology-related  competencies  defined  by  curriculum,  TEA^M  educators  will 
receive  information,  training,  and  support  from  six  primary  resources:  (1)  Knowledge 
Network;  (2)  Teacher  Technology  Exploration  Center  (TTEC);  (3)  Distance  Learning 
Programs;  (4)  Undergraduate  and  Graduate  Coursework;  (5)  Workshops  and  Seminars;  and 
(6)  Site-based  Training  and  Support. 

Knowledge  Network 

The  Knowledge  Network  provides  the  backbone  for  transporting  information  to  and 
among  all  system  stakeholders.  The  network  connects  students,  interns,  public  school 
teachers,  university  faculty,  administrators,  and  business  partners  to  each  other,  to  the 
Teacher  Technology  Exploration  Center,  and  to  experts  and  databases  from  all  over  the 
world.  The  network  allows  the  collaborative  to  bring  expert  knowledge  and  research  on 
innovative  educational  practices  and  technologies  directly  to  the  public  school  sites  as  well 
as  diffuse  the  knowledge  acquired  by  the  collaborative  to  other  researchers  and  educators. 
Connection  to  state,  national,  and  global  networks  helps  free  learning  from  the  confines  of 
the  classroom,  giving  students  and  teachers  virtually  limitless  access  to  knowledge  and 
information  produced  by  students,  teachers,  schools,  businesses,  community  members,  and 
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government  agencies  in  Texas,  the  United  States,  and  beyond.  Figure  3  illustrates  the 
design  of  the  knowledge  network. 

Teacher  Technology  Exploration  Center  (TTEC) 

The  TTEC  s  primary  responsibility  is  to  train  and  empower  students,  teachers,  and 
administrators  to  apply  learning  and  performance  technologies.  The  center  will  allow 
educators  to  explore  leading  edge  technologies  at  their  convenience.  Center  staff  will  guide 
them  in  areas  of  interest,  expand  their  awareness  of  technology,  and  provide  the  knowledge 
and  expertise  required  to  utilize  technology.  Housed  on  the  University  campus,  the  TTEC 
will  he  readily  accessible  to  teachers  and  administrators  from  over  600  schools  in  the  22 
independent  school  districts  surrounding  the  center.  There  will  also  be  an  on-line 
information  service  so  that  people  can  find  out  what  is  currently  available  at  the  center. 

The  TTEC  contains  six  components:  (1)  a  high-tech  electronic  classroom;  (2)  a 
showcase  arena;  (3)  a  multimedia  development  lab;  (4)  a  resource  library;  (5)  video 
production  studio;  and  (6)  research  and  development  lab  (Figure  4).  The  following  is  a  brief 
description  of  each  component. 

Electronic  Classroom.  The  Electronic  Classroom  will  model  a  classroom  of  the  future. 
It  will  include  multi-platform,  electronic  workstations  that  will  support  1-6  users  that  will 
be  networked  to  both  local  and  wide  area  networks,  automated  presentation  stations  and 
capabilities  for  synchronous  and  asynchronous  video  and  audio  conferencing.  Graduate  and 
undergraduate  classes,  workshops,  seminars  will  be  provided  in  the  classroom. 
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Figure  3.  Collaborative-Wide  Knowledge  Network 

Showcase  Arena.  The  Showcase  Arena  features  cutting-edge  educational 
technologies.  Here,  teachers,  students,  administrators,  and  university  faculty  can  explore  the 
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latest  in  educational  hardware  and  software  at  their  own  leisure.  The  arena  showcases  a 
wide  variety  of  vendor  products  and  innovative  applications  developed  by  research 
institutions  such  as  electronic  performance  support  systems,  telecommunication  interfaces, 
electronic  learning  environments,  instructional  management  software  and  administrative, 
presentation,  authoring  and  multimedia  systems. 

Multimedia  Development  Lab.  The  Multimedia  Development  Lab  will  be  a  state  of 
the  art  facility  for  pre-service  and  in 'service  teachers  and  university  faculty  to  receive  training 
and  develop  their  own  multi-media  programs.  The  development  area  would  contain  multiple 
platforms  for  creating  and  conducting  research  on  multimedia  programs,  electronic 
performance  support  tools,  and  innovative  learning  environments.  It  will  facilitate  high-level 
qualitative  and  quantitative  research  and  the  development  of  innovative  instructional 
programs. 

Resource  Library.  The  Technology  Resource  Library  would  house  instructional 
materials,  educational  software,  and  vendor  information  for  educators.  The  library  will 
feature  an  indexed  collection  of  educational  software  and  information  on  instructional 
technology.  The  materials  will  be  made  accessible  both  manually  and  on-line.  Resources  will 
also  include  informative  materials  on  vendor  supported  technologies.  Educators  visiting  the 
center  will  able  to  preview  the  materials  as  well  as  receive  training  on  its  use.  Educators 
will  also  be  allowed  to  check  out  the  materials  so  that  the  can  demonstrate  them  at  the 
collaborating  school  sites. 

Video  Production  Studio.  The  TTEC  Video  Production  Studio  provides  facilities  to 
edit  and  produce  video  tapes.  The  studio  also  serves  as  a  receiving  point  and  distribution 
site  for  video  conferencing,  distance  learning  via  satellite  or  microwave,  and  closed  circuit 
television.  Post- production  facilities  include  the  capability  for  non-linear  digital  editing  or 
video,  voice,  text  and  graphics. 

Research  &  Development  Lab.  The  TTEC  Research  and  Development  Lab  services 
as  a  focal  point  for  research  projects  currently  conducted  by  UHCL  faculty  and  students.  The 
lab  provides  a  synergistic  environment  where  software  engineers,  instructional  technologists, 
graphic  artists,  cognitive  psychologists  and  content  area  specialists  work  together  to  build 
learning  environments  of  tomorrow.  The  lab  contains  the  hardware  and  software  necessary 
to  truly  impact  the  way  we  teach  and  learn  via  cutting-edge  computer  technologies. 
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Figure  4.  Teacher  Technology  Exploration  Center 
Distance  Learning  Programs 

The  development  and  implementation  of  distance  learning  programs  allow  the 
TEA^M  collaborative  to  optimize  its  resources.  Consistent  with  the  TEA^M's  strategy  for 
providing  site-based,  real-world  experiences,  distance  learning  prograrr   ml\  permit 
university  faculty  to  deliver  instruction  to  teachers  and  students  at  the  collaborating  school 
without  the  cost  of  sending  instructors  to  each  site.  In-service  workshops  and  seminars 
provided  by  the  Region  IV  Education  Service  Center  and  by  the  UHCL  Teacher  Center  may 
also  be  delivered  to  each  site  by  this  mechanism. 

In  addition  to  providing  information  and  training  to  TEA^M  collaborative  members, 
the  development  of  the  distance  learning  program  will  allow  students  and  teachers  to 
communicate  and  share  experiences  with  other  students,  teachers,  and  other  professionals 
from  all  over  the  world.  After  the  initial  program  and  networking  infrastructure  have  been 
installed,  UHCL  and  the  Education  Service  Center  will  work  with  teachers  at  each 
collaborating  site  to  develop  their  own  distance  learning  programs.  These  programs  will  help 
break  down  classroom  walls  and  allow  students  at  the  collaborating  sites  to  share 
experiences  with  students  from  other  regions  of  the  Texas,  the  United  States,  and  around 
the  world. 

Development  of  the  distance  learning  programs  will  be  a  collaborative  effort  between 
the  Region  IV  Education  Service  Center,  the  collaborating  schools,  the  University  of  Houston- 
-Clear  Lake,  and  public  utility  companies.  During  the  first  year  of  the  initiative, 
collaborating  schools  will  be  linked  to  the  existing  Interact  system  with  a  TV  antennae,  TV 
monitors,  VCRs  and  telephone  lines.  One-way  video  and  two-way  audio  transmission  will 
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permit  UHCL,  the  Region  IV  Education  Service  Center,  and  the  UHCL  Teacher  Center  to 
deliver  interactive  workshops  and  courses  to  each  school  site.  This  system  will  meet  the 
collaborative's  immediate  needs  as  well  as  provide  a  foundation  for  the  development  of 
future  distance  programs.  In  addition.  Southwestern  Bell  and  Phonoscopes  Inc.  are  now 
assisting  UHCL  assess  the  availability  of  networking  resources  at  the  collaborating  schools 
and  at  the  university  in  preparation. 

The  long-term  plan  is  to  connect  public  schools,  UHCL,  and  the  Region  IV  Education 
Service  Center  with  high  speed  digital  networks.  Digital  networks  reduce  attenuation, 
distortion,  and  noise  accumulation  common  to  current  analog  systems.  Digital  transmission 
is  also  the  universal  language  of  data  communications,  thus,  reducing  compatibility 
problems  and  the  expense  of  converting  analog  and  digital  signals.  Digital  networks  allow 
two-way  video,  audio,  and  data  transmission  between  all  sites  with  greater 

Currently,  the  plan  is  to  keep  abreast  of  legislative,  university,  district,  and  school 
decisions  regarding  technology,  monitor  technological  advances,  and  continue  to  develop  our 
own  capacity  for  delivering  distance  learning  programs.  As  a  clearer  picture  of  this  technology 
emerges,  we  will  make  much  more  informed  decisions  about  the  purchase  and  installation  of 
high-end  networking  systems.  Until  then,  we  will  train  educators  in  the  design, 
development,  implementation,  and  evaluation  of  distance  learning  programs  utilizing  the 
existing  Interact  system  as  well  as  any  system  made  available  in  the  TTEC. 

Undergraduate  and  Graduate  Courses  at  UHCL 

Pre-service  teachers  will  acquire  much  of  the  basic,  technology  related  skills  in 
undergraduate  courses  at  UHCL.  Soon,  all  pre-service  teachers  participating  in  the  TEA^M 
collaborative  will  be  required  to  take  a  computer  literacy  course  as  a  pre-requisite  to 
enrollment  in  the  teacher  education  program.  As  a  part  of  their  teacher  education  program, 
they  be  required  to  take  INST  3133-Classroom  Computer  Usage.  In  this  course,  students 
will  learn  how  to  use  personal  performance  tools,  such  as  word  processors,  databases, 
spreadsheets,  and  communication  applications  and  learn  how  to  locate,  evaluate,  and 
integrate  computer-based  educational  programs.  Students  in  this  class  will  also  learn  how 
to  access  and  critically  examine  research  studies  on  computer  use  in  education,  In  addition, 
pre-service  teachers  will  learn  technology  related  skills  as  a  part  of  their  respective?  teaching 
methods  course.  In  these  courses,  students  learn  how  to  integrate  subject-area  related 
computer-based  educational  program  with  course  curriculum. 

The  University  of  Houston-Clear  Lake  also  offers  a  series  of  courses  and  a  mastei'H 
degree  in  instructional  technology.  As  a  NCATE  accredited  program,  the  Instructional 
Technology  program  at  Clear  Lake  offers  quality  classes  on  the  use  of  leading-edge 
instructional  technologies  (e.g.,  Macromind  Director,  Authorware,  and  hypermedia),  1  he 
systematic  design  of  instruction,  and  needs  assessment  and  strategic  planning.  All  pre- 
service  and  in-service  teachers  involved  with  the  TEA^M  collaborative  will  be  urged  to  take 
these  courses  as  well  as  others  provided  by  UHCL. 

Workshops  and  Seminars 

To  be  responsive  to  the  on  going  needs  of  educators  at  each  of  the  collaborating 
school  sites,  workshops  and  seminars  will  be  given  by  UHCL,  the  UHCL  Teacher  Center, 
and  the  Region  IV  Education  Service  Center.  A  survey  of  educators  will  be  taken  on  a  regular 
basis  to  determine  what  type  of  workshops  and  seminars  are  to  be  delivered.  For  example, 
during  this  past  spring,  a  survey  was  distributed  to  all  educators  at  each  of  the  four 
collaborating  school  sites.  It  was  indicated  that  a  significant  number  of  educators  would  like 
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some  training  and  information  on  the  use  of  TENET.  These  requirements  were  matched 
with  resources  available  at  UHCL,  the  UHCL  Teacher  Center,  and  the  Education  Service 
Center  and  it  was  decided  that  faculty  at  UHCL  will  provide  a  workshop  on  TENET  during 
the  summer.  Workshops  and  seminars  will  be  provided  on  an  as  needed  basis  and  will  be 
delivered  at  the  UHCL  campus,  at  the  Region  IV  Education  Center,  at  one  of  the 
collaborating  school  sites,  or  via  the  distance  learning  network. 

SiteiBasgd  Training  an4  Support 

It  is  anticipated  that  the  integration  of  technology  at  each  of  the  collaborating 
schools  will  require  some  site-based  training  and  support.  No  matter  how  much  instruction 
is  given  through  courses,  workshops,  and  seminars,  there  will  be  some  need,  particularly 
during  the  early  stages  of  this  initiative  for  one-on-one  support.  The  plan  is  to  give  pre- 
service  teachers  sufficient  skills  so  that  they  can  provide  practicing  educators  with  much  of 
the  support  and  training  necessary  to  use  technology.  However,  particularly  during  the  first 
year  of  the  program,  both  pre-service  and  in-service  teachers  may  require  training, 
particularly  in  the  areas  of  networking  and  the  use  of  learning  management  systems. 
Faculty  and  staff  at  UHCL,  graduate  students  in  the  Instructional  Technology  program  at 
Clear  Lake,  and  staff  at  the  Region  IV  Education  Service  Center  will  go  to  each  of  the 
collaborating  school  site  and  provide  training  and  support  on  an  as  needed  basis. 

Assessment 

The  function  of  assessment  is  to  collect  data  on  pre-service  and  in-service  teacher 
performance.  In  terms  of  technology,  data  will  be  collected  to  determine  if  teachers:  (1) 
acquired  the  computer  technology  related  skills  and  knowledge  defined  by  curriculum;  and 
(2)  apply  these  skills  and  knowledge  to  facilitate  their  students  and  their  own  performance. 
Data  from  assessment  will  be  used  as  feedback  for  pre-service  and  in-service  teachers  to 
improve  their  performance  as  well  as  to  evaluate  the  effectiveness  and  efficiency  of  the 
technology  system. 

For  the  TEA^M  collaborative,  pre-service  teachers  achievement  of  computer- 
technology  related  outcomes  will  be  assessed  by  a  portfolio.  For  their  portfolio,  pre-service 
teachers  are  expected  to  provide  five  different  types  of  products  as  evidence  of  their 
accomplishments.  The  products  include:  (1)  research,  such  as  a  review  of  literature  on  the 
use  of  particular  computer  application  in  education;  (2)  a  video  which  depicts  the  teacher 
implementing  computer-technology  in  an  instructional  setting;  (3)  lesson  plans  which  show 
the  inclusion  of  technology;  (4)  assessments  that  demonstrate  teachers  ability  to  measure 
students  acquisition  and  application  of  computer-technology;  and  (5)  resources,  such  as  a 
directory  of  people,  software,  and  hardware  products  and  services.  The  portfolios  are 
examined  by  university  faculty  as  well  as  the  TEA^M  of  teachers  who  are  supervising  the 
pre-service  teacher. 

Evaluation 

The  evaluation  function  monitors  both  internal  programs  and  external  conditions  to 
make  necessary  revisions  to  the  system.  This  approach  assumes  that  if  teachers  and 
students  are  not  learning,  one  or  more  components  of  the  system  is  not  working.  Where 
assessment  focused  on  individuals  acquisition  and  application  of  specified  learning 
outcomes,  evaluation  aggregates  assessment  data  and  collects  additional  information  to 
determine  the  effectiveness  and  efficiency  of  the  system  and  system  components. 
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A  framework  was  created  to  guide  the  evaluation  process.  Based  on  Kaufman's 
(1983)  Organizational  Elements  Model,  the  framework  outlines  5  different  levels  of 
organizational  effort  and  results  that  are  to  be  evaluated.  The  framework  also  helps  define 
evaluation  questions  and  specific  components  that  are  to  be  evaluated.  Figure  5  illustrates 
the  evaluation  framework  used  to  guide  the  evaluation  of  the  TEA^M  system  for  infusing 
computer  technology. 

Logistics 

Logistics  are  responsible  for  all  business-related  aspects  schooling.  In  terms  of 
technology,  this  refers  directly  to  the  purchase  and  maintenance  of  hardware  and  software. 
Grants  are  often  written  to  cover  the  initial  cost  of  buying  and  installing  computer 
equipment.  However,  relatively  little  thought  goes  to  the  institutionalization  of  hardware 
and  software  upgrades  and  maintenance.  Educational  innovations,  pai  icularly  those 
received  from  grants,  are  frequently  viewed  as  add  ons  to  the  current  system.  Thus,  support 
for  certain  innovations  dissolves  as  political  and  economic  climates  change.  Without 
continued  funding,  hardware  and  software  soon  become  obsolete  and  are  left  to  gather  dust 
in  an  obscure  corner  of  the  classroom. 

To  address  these  concerns,  the  TEAM  collaborative  works  directly  with  each  school's 
district  office.  Decisions  regarding  hardware  and  software  purchases,  networking,  and 
maintenance  are  aligned  directly  with  each  districts  long-term  plan  for  integrating  computer- 
technology.  In  this  manner,  the  collaborative  ensures  that  hardware  and  software  brought  in 
through  the  grant  will  be  compatible  with  the  technologies  to  be  installed  at  each  site  by  the 
district  over  the  next  five  years. 

Based  on  the  premise  that  schools  will  continue  to  acquire  classroom  computers,  the 
TEAM  collaborative  decided  to  allocate  its  technology  resources  on  computer  workstations  for 
each  teacher's  desk.  If  teachers  are  expected  to  develop  innovative  strategies  for  increasing 
students'  performance  through  the  use  of  computer  technology,  it  is  believed  that  they  must 
first  feel  confident  in  their  ability  to  use  it  through  its  daily  application. 
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Figure  5,  TEA^M  Evaluation  Framework 


Each  public  school  received  twenty  teacher  workstations  (one  per  teacher  working 
directly  with  the  TEAM  collaborative)  and  a  multimedia  presentation  system.  One  school 
choose  IBM  and  another  IBM  compatibles  because  the  district  would  suppoit  only  those 
platforms.  The  third  school  choose  Macintoshes  because  they  felt  that  it  would  be  easier  for 
their  students  and  teachers  to  learn.  The  IBM  and  IBM  compatible  teacher  workstations 
included  a  33Mhz  486DX  processor,  4-8MB  of  RAM,  120-240MB  hard  drives  with  DOS  and 
Windows  pre- installed,  SVGA  monitors,  mouse,  keyboard,  and  lObaseT  transceivers. 
Centris  BIO  with  CD  ROM,  8MB  of  RAM  and  240MB  hard  drives  and  Duo  Docks  with 
PowerBook  Duo  210  with  4MB  of  RAM  and  80MB  hard  drives  were  purchased  for  the 
Macintosh  school.  Each  multimedia  presentation  system  included  a  high  powered  CPU,  color 
scanner,  a  CD  and  a  laser  disc  player,  a  LCD  projection  panel,  and  an  overhead  projector. 

By  working  with  each  district  office,  the  collaborative  was  also  able  to  leverage 
resources  to  install  the  local  area  networks.  In  most  cases,  plans  were  already  in  place  for 
cabling  the  entire  sch<     within  the  next  two  to  three  years.  By  pooling  the  resources,  the 
collaborative  was  able  to  get  the  district  to  install  the  cabling  required  to  network  the 
teacher  workstations  by  the  end  of  this  school  year.  The  public  schools  will  retain  the 
computer  equipment  as  long  as  they  remain  active  participants  of  the  collaborative  (e  g., 
provide  internships  for  TEAM  pro-service  teachers)  and  the  districts  assume  responsibility 
for  maintenance  and  upgrades. 


Management  Operations 


Management  operations  is  charged  with  making  decisions  regarding  organizational 
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policies  and  procedures,  resource  allocation,  and  program  design  and  implementation.  In 
education,  researchers  and  administrators  distant  from  daily  classroom  practices,  initiate 
and  manage  numerous  grant-related  programs.  Few  top-down  strategies,  however,  have  had 
a  significant  impact  on  classroom  instruction  because  they  fail  to  account  for  the  pragmatics 
of  daily  school  routine.  Scorned  by  teachers  who  have  been  asked  repeatedly  to  implement 
new  programs  with  little  time  or  training,  top-down  interventions  are  not  owned  by  the 
practitioners  and  thus,  are  not  frequently  implemented  as  planned. 

New  programs  designed  by  site-based  eomn  ttees  have  also  seen  limited  success.  In 
contrast  to  top-down  strategies,  "bottom-up"  initiati  .  c  are  constrained  by  the  lack  of  time 
and  training  for  program  design.  As  advances  in  technology  continue  to  increase,  even  the 
most  avid  technologists  arc  hard  pressed  to  keep  up  with  the  latest  changes.  Practitioners 
responsible  for  educating  20-40  students  on  a  daily  basis  can  not  be  expected  to 
continuously  acquire  the  skills  necessary  to  apply  leading-edge  technologies  without 
additional  resources. 

To  address  the  limitations  inherent  to  top-down  and  bottom-up  interventions,  TEAM 
employs  a  collaborative  consensus  process  to  guide  system  design  and  implementation.  In 
this  process,  representatives  from  all  system  stakeholders  participate  in  programmatic 
decision-making. 

Five  committees  manage  the  computer-technology  system  for  the  TEAM 
collaborative,  including:  (1)  three  site  committees,  made  up  of  the  principal,  the*  site 
coordinator,  the  school  computer  lab  coordinator,  and  the  district  technology  coordinator  for 
each  school;  (2)  a  university  committee,  consisting  of  three  representatives  from  the 
University  Computing  department  and  three  faculty  members  from  the  Instructional 
Technology  program;  and  (3)  a  collaborative-wide  technology  committee,  comprised  of  all 
.  members  of  the  public  school  and  university  committees.  In  addition,  a  Director  of 
Technology  for  the  TEAM  collaborative  is  charged  with  coordinating  the  activities  of  the 
various  committees  as  well  as  integrating  technology  with  other  aspects  of  the  TEAM's 
teacher  education  program. 

The  site-based  committees  are  responsible  for  making  decisions  that  affect  resource 
allocations  and  program  implementation  at  each  school.  For  instance,  each  public  school 
committee  determined  the  type  of  computers  that  were  to  be  provided  by  the  collaborative. 
The  university  committee,  in  comparison,  choose  the  hardware  and  software  used  to 
network  the  schools  to  the  university  and  created  the  Teacher  Technology  Exploration 
Center.  The  collaborative-wide  committee  set  policies  for  allocating  grant-related  funds  and 
specify  the  software  to  be  purchased  and  supported  by  the  collaborative. 

Professional  Development 

The  function  of  professional  development  is  to  train  and  support  staff  in 
implementing  the  TEAM  system  for  infusing  computer  technology.  The  products  of 
professional  development  are  educational  experiences  and  staff  support  services.  Initially, 
focus  is  being  placed  on  developing  each  site's  capacity  to  manage  their  own  local  area 
network  and  utilize  the  computer  applications  provided  by  the  collaborative,  The  strategy  is 
to  develop  a  core  of  technology  proficient  educators  who  can  trouble  shoot  basic  problems  and 
serve  as  a  resource  for  students,  teachers,  and  administrators  at  each  site. 

Educational  experiences  range  from  specific  training  sessions  provided  by  computer 
vendors  on  the  use  of  their  equipment,  to  more  general  types  of  experiences  such  as 
attending  computer  conferences.  The  goal  is  to  provide  a  wide  range  of  opportunities  that 
will  enable  educators  to  pursue  their  own  areas  of  interest.  In  this  manner,  educators  will 
gain  expertise  in  different  areas  of  computer  use  at  each  collaborating  school  site  and  will  bp 
able  to  provide  information  and  support  to  other  educators  at  their  school  on  a  timely  basis. 
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Particular  attention  is  also  being  placed  on  establishing  a  collaborative-wide  human 
performance  support  system.  The  support  network  identifies  individuals  from  the  university, 
public  schools,  computer  vendors  and  other  businesses  both  within  and  outside  of  the 
collahorative  who  have  expertise  in  particular  areas  of  computer  use  such  as  networking, 
desktop  publishing,  distance  learning,  and  multimedia  development.  A  list  of  experts, 
including  their  e-mail  address  and  phone  number,  is  kept  up-to-date  and  disseminated  to 
all  members  of  the  collaborative. 

Summary 

The  TEA^M  collaborative  is  currently  in  its  first  year  of  development.  The  goals  for 
technology  during  this  year  are  to:  (1)  analyze  the  computer-technology  requirements  at  each 
collaborating  site;  (2)  install  computer  workstations,  local  area  networks,  and  the  wide  area 
network  which  links  the  public  schools  to  the  University,  TENET,  and  Internet;  (3)  purchase 
and  install  basic  computer  applications  (e.g.,  productivity  &  telecommunication  tools);  (4) 
define  a  computer-technology  curriculum  for  both  pre-service  and  in-service  teachers;  and  (5) 
begin  the  design  and  delivery  of  instruction.  To  date,  the  technology  requirements  at  each 
collaborating  site  have  been  assessed,  curriculum  and  instruction  are  being  designed,  and 
the  computer  workstations  have  been  purchased  and  installed.  By  this  summer,  the  local 
area  networks,  the  collaborative-wide  network,  and  the  Teacher  Technology  Exploration 
Center  should  be  fully  functional.  By  year  two,  we  hope  to  see  a  significant  number  of  pre- 
service  and  in-service  teachers  who  are  able  to  increase  student  and  their  own  performance 
through  the  inf  jsion  of  computer-technology. 
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Arkansas  is  a  rural  state  with  diverse  educational  needs.  For  many  years  Arkansas 
ranked  50th  in  the  nation  by  most  criteria  used  in  measuring  educational  comparisons  with 
other  states.  Governor  Bill  Clinton  recognized  that  in  order  for  Arkansas  to  improve 
economically  it  must  invest  in  education  and  measures  were  implemented  during  his  tenure 
to  improve  education.  Additional  monies  were  raised  to  increase  teacher  salaries,  build 
facilities  and  expand  curricula.  Another  major  change  in  education  was  the  establishment  of 
standards  (1983)  for  all  schools  in  Arkansas.  Course  content  guides  were  developed  for 
every  subject  area  and  documentation  was  required  for  minimum  performance  in  basic 
skills.  Students  were  tested  in  the  third,  sixth  and  eighth  grades.  Although  most  of  what 
happened  was  positive,  there  were  some  negatives.  Instead  of  the  curriculum  driving  the 
content  and  instruction,  the  test  determined  the  curriculum.  Teacher  expertise  and 
knowledge  of  student  management,  group  dynamics  and  flexibility  were  lost.  Teaching  to 
the  test  occurred  in  many  schools;  when  test  results  were  published  in  local  newspapers, 
this  put  even  more  emphasis  on  test  results. 

In  1991  Arkansas  passed  Act  236  which  required  public  schools  to  restructure. 
Learner  outcomes  were  developed  which  correlated  with  the  six  national  education  goals. 
Local  schools  are  currently  working  to  finalize  frameworks  which  will  be  implemented  over 
the  next  few  years.  One  component  of  Act  236  was  the  establishment  of  the  Arkansas 
Leadership  Academy  with  four  areas  of  concentration:  (1)  Leadership  Development;  (2) 
District  Support;  (3)  Minority  Recruitment;  and,  (4)  Information,  Research  and  Technology. 
The  Arkansas  Leadership  Academy  has  many  partners  across  the  state:  colleges  and 
universities;  K-12  schools;  Arknasas  Public  School  Computer  Network  (APSCN);  Wal-Mart; 
Tyson;  Arkansas  Educational  Television  Network  (AETN);  Arkansas  Library  Association 
(ALA);  Arkansas  Department  of  Education  (ADE);  Arkansas  Association  of  Instructional 
Media  (AAIM);  Instructional  Microcomputer  Project  for  Arkansas  Classrooms  (IMPAC);  and 
private  individuals.  The  Academy  has  within  its  structure  the  mechanism  to  involve  all  the 
players  to  truly  change  education  in  Arkansas. 

The  decisions  concerning  technology  in  Arkansas  have  been  made  by  people  working 
in  isolation.  Project  IMPAC  labs  have  served  a  useful  purpose  and  are  designed  in  such  a 
way  as  to  furnish  training,  technical  suport  and  maintenance.  AETN  provides  videos  to 
education  cooperatives  and  schools.  APSCN  has  the  task  of  networking  all  320  school 
districts  across  the  state  for  administrative  reporting  and  also  for  instructiona1  uses.  There 
are  pilot  schools  that  have  built  their  local  network  and  are  exploring  the  opportunities  of 
accessing  information  electronically  for  instructional  use.  All  of  these  players  are  to  be 
complimented  on  their  efforts  and  support,  but  Arkansas  schools  must  utilize  this 
collaborative  effort  to  make  a  lasting  change  in  Arkansas  schools.  We  are  currently  looking 
at  restructuring  not  only  what  we  teach,  but  how  we  teach  it. 

With  these  ends  in  mind,  The  Arkansas  Academy  for  Leadership  Training  and 
School-Based  Management  Technology  Management  Technology  Committee  made  the 
decision  to  survey  all  public  school  library  media  specialists  (LMS)  in  Arkansas  during  the 
Spring  of  1993.  The  committee  felt  that  library  media  specialists  are  the  primary 
information  specialists  in  the  schools  with  expertise  in  the  area  of  technology. 

The  purpose  of  the  survey  was  to  determine  what  technology  is  presently  being  used 
in  Arkansas  schools,  to  develop  a  database  of  educators  with  expertise  in  specific  areas  of 
technology,  and  to  compile  a  list  of  priorities  library  media  specialists  would  like  to  have 
implemented  in  their  library  media  centers/schools.  Survey  questions  were  developed  with 
the  assistance  of  the  state  library  supervisor,  library  media  specialists  in  Northwest 
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Arkansas,  the  media  specialist  at  the  Northwest  Arkansas  Education  Service  Cooperative, 
and  the  Arkansas  Academy  for  Leadership  Training  and  School-Based  Management 
Technology  Committee,  The  University  of  Arkansas  Sociology  Department's  newly  acquired 
Scatitron  survey  equipment  was  used  to  compile  data. 

The  development  of  the  survey  began  in  February,  1993.  Team  players  helped 
develop  a  50-question  survey  instrument  covering  three  major  areas:  district  concerns, 
school  concerns,  and  LMS  concerns.  The  survey  was  mailed  to  967  public  school  LMS  on 
three  separate  occasions  over  a  four  month  period.  In  addition,  LMS  attending  their 
summer  conference  and  the  Arkansas  Library  Association's  Annual  Conference  were 
encouraged  to  take  surveys  back  to  colleagues  who  had  not  responded.  Mr.  Frank  Holman 
wrote  letters  to  superintendents  in  districts  which  had  not  responded  well  to  the  mailings. 
Six  months  after  the  initial  mailing,  a  total  of  845,  or  87%  of  the  Arkansas  public  school 
library  media  specialists  had  returned  a  completed  survey.  Of  the  75  counties  in  Arkansas, 
the  response  rate  was: 

100%  21  counties 

90-99%  10  counties 

80-89%  19  counties 

70-79%  15  counties 

60-69%  7  counties 

50-59%  2  counties 

40-49%  1  county 

A  breakdown  of  numbers  of  counties  reaching  100%  response  by  geographic  region: 


Ouachita  Region  1%  (1  out  of  10  counties) 

Delta  Region  19%  (4  out  of  21  counties) 

Southwest  Region  30%  (6  out  of  20  counties) 

Ozarks  Region  46%  (1 1  out  of  24  counties) 


District  Data  Statewide 

District  survey  results  showed  nearly  two-thirds  of  schools  did  not  have  a  district 
technology  planning  committee  and  71%  did  not  have  a  technology  coordinator.  Planning 
ahead  was  not  being  done  in  69%  of  schools  because  there  weren't  any  electrical 
specifications  for  computer  networks  for  new  buildings.  Financial  support  was  a  problem  as 
well.  Ninety-two  percent  of  schools  have  not  passed  millage  to  support  technology,  and  87% 
have  not  made  plans  for  future  needs  in  technology. 

School  Data  Statewide 

Data  collected  revealed  that  68rj  of  the  states  school  library  media  centers  are  not 
automated.  Of  the  centers  which  are  automated,  45%  use  Follett's  automation  system. 
Fifty-four  percent  of  the  schools  have  satellite  systems  but  only  about  a  third  of  those 
systems  are  being  used.  Ninety  percent  of  the  schools  are  not  involved  in  distance 
education.  Types  of  computers  being  used  in  the  schools  are:  Apple  lie  -  39%;  Macintosh  - 
10%;  IBM/compatible  -  42%;  other  -  9%.  [This  reveals  a  problem  in  that  Apple  lie's  are  not 
compatible  with  the  state-wide  computer  network  system,]  IMPAC  has  installed  labs  in 
52%  of  Arkansas  schools  during  the  past  ten  years.  Only  about  one  in  five  schools  is  using 
networks  and  most  of  those  use  the  Novell  system.  Library  media  specialists  reported  that 
in  53%  of  the  schools,  no  more  than  10%  of  the  classrooms  are  equipped  with  computers  and 
almost  none  are  networked.  When  asked  if  teachers  had  access  to  specific  technology  the 
following  responses  were  tabulated: 
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FAX  machine  access 
ARK  FAX  Network 
ARKnet/INTER/net 
Email/FredMail 


Modem  access 


Channel  1/Whittle 


Yes 
18% 
23% 
3% 
5% 
4% 
28% 


82% 
77% 
97% 
95% 
96% 
70% 


Arkansas  public  schools  are  using  cable  television  (61%),  multimedia  (74%),  CD- 
Roms  (39%),  and  laser  video  technology  (19%).  Computer  use  in  schools  is  primarily  for 
special  programs  such  as  Chapter  1,  special  education,  and  gifted  and  talented,  but  not  in 
the  regular  classroom. 

Library  Media  Specialists  Data  Statewide 

Sixty-one  percent  of  LMS  have  earned  a  masters  degree.  Nearly  one-third  reported 
receiving  their  library  media  education  at  the  University  of  Central  Arkansas,  Conway, 
Arkansas,  while  17%  received  their  training  at  the  University  of  Arkansas,  Fayetteville. 
When  queried  as  to  whether  they  would  like  to  have  an  American  Library  Association 
accreditation  program  in  Arkansas,  96%  said  'Yes."  Almost  one-half  of  the  LMS  indicated 
they  worked  under  a  nine- month  contract,  the  same  as  classroom  teachers.  A  larger 
percentage  (63%)  reported  not  having  clerical  assistance  in  the  library  media  center.  Sixty 
percent  of  Arkansas  school  districts  have  evaluation  forms  designed  for  LMS.  Eighty  percent 
of  the  LMS  said  they  would  like  to  have  a  specific  amount  of  their  budget  allocated  for 
technology.  A  full  61%  did  not  support  the  concept  of  year  'round  school.  There  was  an 
almost  even  split  regarding  flexible  scheduling. 

When  LMS  were  asked  to  indicate  areas  in  which  they  felt  they  had  expertise,  47% 
indicated  computer  technology  and  30%  mentioned  multimedia  presentations.  When  asked 
if  they  were  willing  to  share  this  expertise  with  others,  66%  said  "No."  The  researchers  are 
currently  processing  this  information  in  order  to  find  those  willing  to  share  their  areas  of 
technical  expertise.  This  will  aid  in  the  creation  of  workshops  to  be  offered  throughout  the 
state. 

LMS  were  also  asked  to  list  at  least  five  areas  they  would  like  to  have  implemented. 
The  survey  placed  responses  in  groups  of  five  due  to  the  constraints  of  the  survey  answer 
form,  making  it  difficult  to  determine  if  responses  were  by  groups  or  with  the  entire  25  being 
considered.  This  information  is  currently  being  analyzed.  By  percentage  responding,  the  top 
five  areas  were:  budget  concerns;  flexible  scheduling;  technology  seminars  for  LMS; 
networking  with  LMS;  and  clerical  assistance  for  LMS.  The  areas  least  marked  were: 
library  media  certification;  censorship;  teleconferences;  satellite  systems;  and  Channel 
1/Whittle. 
Summary 

Arkansas  is  indeed  "The  Comeback  State."  Watch  as  the  Arkansas  Leadership 
Academy  and  its  many  partners  utilize  technology  in  the  classroom.  Team  members 
working  together  will  provide  the  best  possible  educational  opportunities  for  all  students, 
and  technology  will  play  a  vital  role  at  all  levels  of  the  instructional  process.  This  survey 
will  assist  in  showing  Arkansas  public  schools  the  direction  they  need  to  go  to  reach  their 


This  completes  Phase  1  of  the  survey  -  analyzing  the  data  statewide.  Phase  2  will 
compare  data  by  the  geographic  regions  of  Ouachita,  Delta,  Southwest,  and  Ozarks. 
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Persistence  and  Small  Group  Interaction 
One  approach  to  designing  computer-based  instruction  (CBI)  involves  using  the 
adaptive  capabilities  of  the  computer  to  provide  individualize  instruction  (Kinzie,  1990; 
Carrier  &  Jonassen,  1988;  Tennyson  &  Park,  1987;  Reiser,  1987).  However,  practical 
limitations,  such  as  the  extensive  time  requirements  and  associated  financial  costs,  limit 
the  potential  for  individualizing  CBI  (Carrier  &  Jonassen,  1988,  Bork,  1987).  Another 
limiting  factor  is  the  shortage  of  computers  and  subsequent  logistical  constraints  endured  by 
most  schools.  In  practice,  most  students  work  in  groups,  not  alone,  at  the  computer  (Becker, 
1986;  Hooper,  1993). 

Focusing  on  individualized  CBI  also  ignores  consistent  research  findings  that 
demonstrate  the  effectiveness  of  cooperative  learning  methods  on  student  achievement,  their 
attitudes  towards  instruction,  and  other  students  (Johnson,  Johnson,  &  Maruyama,  1983; 
Johnson  &  Johnson,  1991;  Slavin,  1991).  Furthermore,  a  growing  body  of  evidence  has 
accumulated  which  indicates  similar  benefits  for  completing  CBI  in  cooperative  learning 
groups  (Johnson,  Johnson,  &  Stanne,  1985,  1986;  Dalton,  Hannafin,  &  Hooper,  1989; 
Hooper,  1992;  Light  &  Blaye,  1990;  Nastasi  &  Clements,  1991;  Jackson,  Fletcher,  & 
Messer,  1992). 

From  a  design  perspective,  however,  the  pertinent  issue  no  longer  appears  to  be 
whether  cooperative  CBI  benefits  student  achievement,  but  rather  involves  identifying 
factors  which  affect  group  performance.  Perhaps  the  most  important  determinants  of  group 
performance  are  the  quality  and  quantity  of  student  interaction.  Webb  (1982b,  1985)  found 
that  student  achievement  benefits  from  frequent  intra-group  interactions  which  include  both 
the  giving  and  receiving  of  elaborated  explanations  which  go  beyond  mere  "terminal  help".  It 
is  important,  therefore,  to  recognize  that  the  nature  of  intra-group  interaction  is  likely  to  be 
a  direct  reflection  of  the  internal  dynamics  within  the  group. 

One  factor  that  appears  to  influence  the  nature  of  interaction  is  ability  grouping, 
Webb's  research  indicates  the  positive  achievement  affect  that  heterogeneous  ability 
grouping  has  on  high-  and  low-ability  students,  but  not  average  students  (1982a,b).  In 
Webb's  studies,  when  grouped  heterogeneously,  average  ability  students  were  frequently 
excluded  from  the  peer  tutoring  relationships  that  often  existed  between  high-  and  low- 
ability  students.  In  other  studies,  heterogeneous-ability  grouping  benefitted  the 
performance  of  low-ability  students,  but  high-ability  students  did  best  when  grouped 
homogeneously  (Hooper  &  Hannafin,1988;  Hooper,  1991). 

Ability  grouping  influences  the  quantity  and  quality  of  interaction.  Low-ability 
students  interact  more  frequently  and  effectively  when  grouped  heterogeneously  by  ability, 
but  high-ability  students  interact  equally  effectively  in  homogeneous  and  heterogeneous 
ability  groups.  The  rate  of  interactions  decreases  for  low-ability  students  who  are 
homogeneously  grouped  (Hooper  &  Hannafin,  1991),  but  increases  for  high-ability  students 
who  are  homogeneously  grouped  (Hooper,  1992). 

In  addition  to  ability  grouping,  several  other  individual  factors  or  personality 
characteristics  may  affect  the  intra-group  dynamics.  One  such  factor  is  a  student's  task 
persistence  level.  In  a  study  of  the  affects  of  learner  control  on  student  performance,  Carrier 
&  Williams  (1988)  found  a  curvilinear  relationship  between  an  individual  student's  task 
persistence  and  subsequent  achievement.    Medium  task  persisters  demonstrated  higher 
achievement  than  both  extremely  low  and  high  persisters,  When  given  control  of  the  amount 
and  nature  of  instruction  and  practice  to  receive  during  CBI,  high  persisters  persevered  even 
though  they  know  the  material;  their  persistence,  similar  to  the  effects  seen  in  high 
motivation  or  arousal  (Yerkes  &  Dodson,  1908)  apparently  inhibited  their  learning.  Low 
persisters  stopped  studying  the  lesson  material  prematurely,  believing  that  they  had 
sufficiently  mastered  the  lesson  content. 
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To  date,  however,  the  effects  on  performance  of  grouping  students  according  to 
individual  variations  in  persistence  levels  have  not  been  investigated.  In  the  present  study 
we  examined  how  homogeneous  and  heterogeneous  groupings  of  individual  persistence  levels 
affected  group  performance  and  interaction  patterns.  We  were  interested  in  whether  the 
presence  of  an  average  persister  would  enhance  the  performance  of  the  high  or  low 
persisters,  or  conversely  whether  high  or  low  persisters  would  detract  from  the  performance 
of  the  average  persisters.  We  were  also  interested  in  the  affect  of  grouping  on  two  other 
outcomes:  students'  attitudes  and  intra-group  interaction.  In  particular,  we  were  interested 
in  examining  the  impact  of  collaboration  on  students'  selection  of  cooperative  learning 
partners  and  in  examining  how  student  interaction  varied  as  a  function  of  group 
composition. 

Methods 

Subjects 

Participants  in  the  study  were  139  sixth  grade  students  enrolled  in  five  classes  at  a 
Midwestern  public  school.  Of  the  total  number,  68  students  were  male  and  71  were  female. 
One  student  was  not  paired,  and  was  removed  from  the  data  being  analyzed,  leaving  138 
students  in  the  study. 
Materials 

The  computer-based  lesson  and  posttest  were  based  on  those  used  in  previous 
research  by  Carrier  and  Williams  (1988).  The  lesson  content  introduces  four  advertising 
concepts:  bandwagon,  uniqueness,  testimonial  and  transfer.  The  lesson  begins  with  a 
concept  definition  followed  by  a  set  of  expository  examples  and  nonexamples.  Next, 
students  attempt  practice  questions  and  choose  whether  to  receive  related  feedback 
indicating  the  accuracy  of  their  responses  and  elaborated  feedback  emphasizing  the  presence 
or  absence  of  critical  attributes.  Students  also  decide  whether  to  attempt  additional 
practice  items.  The  original  materials  were  modified  to  include  a  maximum  of  six  questions 
for  each  concept.  Thus,  for  each  concept  students  could  select  up  to  18  options  including 
examples,  feedback  and  elaboration. 

The  lessons  were  presented  in  three  segments.  The  first  concept  was  used  to  assess 
students'  levels  of  persistence.  This  lesson  was  completed  individually  by  all  students. 
Concepts  two  and  three  were  presented  on  the  second  day  and  were  completed  in 
cooperative  groups.  Concept  four  was  presented  on  the  third  day  and  was  again  completed 
in  groups. 

The  podttests  were  designed  to  measure  students'  achievement  and  attitudes 
toward  their  partners.  The  achievement  posttest  includes  two  parts:  twenty-five  questions 
measure  comprehension  of  the  lesson  material  and  four  problem  solving  questions  measure 
higher  levels  of  cognitive  processing.  Comprehension  questions  involve  classifying  instances 
of  the  lesson  concepts.  Problem  solving  questions  involve  generating  instances  of  lesson 
concepts.  Coefficient-alpha  reliability  for  the  posttest  was  .91.  The  Partner  Preference 
Questionnaire  asks  students  to  list  the  names  of  up  to  five  students  from  their  class  class 
with  whom  they  would  like  to  work. 

The  interaction  rate  was  determined  by  dividing  the  total  number  of  instances  of 
cooperative  behavior  by  the  total  time  an  individual  was  observed.  Thus,  the  interaction 
rate  represents  the  average  number  of  interactions  per  minute.  Interaction  was  observed 
and  coded  by  trained  raters.  The  number  of  student  groups  active  during  each  session  was 
greater  than  the  number  of  trained  raters  available.  Consequently,  we  observed  all  groups 
by  rotating  raters  during  the  lesson.  Raters  rotated  every  five  minutes  to  reduce  coding 
variation.  Furthermore,  approximately  40%  of  the  groups  were  videotaped  during  the 
instruction  to  facilitate  a  qualitative  analysis. 
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Before  the  study,  raters  were  trained  in  observation  and  coding  techniques.  Rater 
training  consisted  of  three  stages.  First,  the  lesson  was  videotaped  with  a  comparable 
group  of  students.  The  videotapes  of  the  pilot  session  were  used  to  develop  the  coding 
instrument.  The  second  stage  of  rater  training  involved  training  eight  graduate  students 
who  had  volunteered  to  be  raters  during  the  study.  The  raters  were  trained  using  a 
combination  of  discussion,  tape  review,  and  practice.  During  practice,  individual  rating 
differences  led  to  review  of  tape  segments  and  group  discussion.  The  third  stage  involved 
testing  raters  performance  to  confirm  consistency  in  the  ratings  given  by  the  observers.  The 
individual  who  conducted  a  post  hoc  videotape  analysis  was  trained  using  similar  processes. 
Procedures 

Students  completed  a  90  minute  workshop,  lead  by  the  investigators,  before  the 
study.  The  purpose  of  the  workshop  was  to  enhance  intra-group  interaction  and 
cooperation.  The  activities  employed  in  the  training  session  are  outlined  elsewhere  (Hooper, 
Temiyakarn,  &  Williams,  1993).  One  week  later,  all  students  completed  a  computer  based 
lesson  alone.  The  purpose  of  the  lesson  was  to  assess  student  persistence.  Based  on  the 
total  number  of  options  selected  during  the  lesson,  students  were  classified  as  High  (H), 
Average  (A),  or  Low  (L)  in  terms  of  persistence. 

Before  working  in  groups,  students  completed  the  Partner  Preference  Questionnaire. 
Next,  dyads  were  formed  by  randomly  assigning  students  within  each  class  to  a  dyad 
resulting  in  six  possible  combinations  of  persistence  (HH,  HM,  HL,  MM,  ML  and  LL). 
Students  completed  the  remaining  three  computer  based  lessons  in  groups  over  two 
successive  days.  One  week  later  students  completed  the  posttest  and  the  Partner 
Preference  Questionnaire. 
Design  and  Data  Analysis 

Dependent  measures  included  Interaction  Rate,  Achievement  (divided  into  Concept 
Learning  and  Problem  Solving),  and  Attitudes.  A  MANOVA  was  conducted  using 
Interaction  Rate  and  the  two  achievement  measures  as  dependent  variables.  Independent 
variables  included  Persistence  Level  (High,  Average,  and  Low),  Gender  (Male  and  Female), 
and  Mix  (Heterogeneous  and  Homogeneous).  Significant  overall  effects  were  followed  up  with 
univariate  ANOVA's.  A  separate  2X2  Chi  square  analysis  was  conducted  on  the  Partner 
Preference  Questionnaire.  Independent  variables  were  Time  (Pretest  and  Posttest)  and 
Partner  Selection  (Yes  or  No).  Calculations  were  made  using  Svstat  (€)  1990,  SYSTAT, 
Inc.),  and  Testat.  (©  1986,  SYSTAT,  Inc.).  All  tests  of  significance  adopted  an  alpha  level  of 
.05. 

Results 

Posttests. 

Table  1  reports  the  means  and  standard  deviations  on  the  posttests.  A  MANOVA 
on  these  data  indicated  significant  effects  for  Persistence  Level  [Wilks1  Lambda  =  .883', 
£(6,226)=  2.42,  p<.02]  and  for  Gender  [Wilks1  Lambda  =  .933,  E(3,113)=  2.70,  d<  ).05.  No 
other  effects  were  statistically  significant. 

Univariate  follow-up  tests  indicated  that  Persistence  Level  was  significantly  related 
to  Interaction  Rate  [MSError=  1-653,  £(2,115)=  4.71,  xF-011].  Average  persisters  interacted 
significantly  more  that  high  and  low  persisters  (see  means  on  Table  1). 


Follow-up  examination  of  the  Gender  effect  indicated  that  Gender  was  significantly 
related  to  Concept  Learning  [MSError=  43.57,  £(1,115)=  4.55,  d=.035]  and  to  Problem 
Solving  [MSError=  1-87,  £(1,115)=  6.84t  p=. 010].  In  each  case,  males  outperformed  females 
(Concept  Learning  15.81  vs.  13.23;  Problem  Solving  2.52  vs.  1.80). 


Insert  Table  1  about  here 


Interaction 

A  pest  hoc  analysis  of  videotapes  recorded  during  the  experiment  was  conducted  to 
validate  the  rating  procedure  conducted  during  the  experiment.  During  the  study  all  groups 
were  rated  by  trained  observers,  but  the  post  hoc  analysis  was  conducted  on  only  a  subset  of 
the  groups.  Comparison  of  these  data  indicated  a  strong  relationship  between  the  social 
interaction  ratings  that  occurred  during  and  after  the  study  (x  =  0.75,  u  <  .001). 
Attitudes 

The  Partner  Preference  Questionnaire  showed  a  significant  increase  in  the  number  of 
students  who  included  their  partner  (Chi-Square  =  37.0,  &  <  01).  Before  the  experiment, 
eighteen  students  included  their  eventual  partners  on  the  Partner  Preference  Questionnaire. 
Following  the  experimental  treatments,  twenty-three  more  students  listed  their  assigned 
partners  as  a  preferred  partner  while  only  two  people  removed  their  partner  from  the  list. 
In  one  of  the  two  cases,  in  which  the  partner  was  initially  listed  but  was  not  listed  following 
the  experimental  treatment,  notes  recorded  during  the  experiment  indicate  that  strong 
negative  social  interaction  occurred  between  the  partners  at  the  beginning  of  the  first  day. 
Correlation  Analyses 

Correlations  between  Interaction  Rate  and  overall  achievement  were  analyzed  to 
determine  the  relationship  between  collaboration  and  achievement  (r  =  -20,  p  =  024). 
Furthermore,  correlations  between  Interaction  Rate  and  overall  achievement  were  further 
examined  to  determine  whether  the  relationship  varied  according  to  Persistence  Level.  This 
analysis  indicated  that  the  relationship  was  significant  only  for  the  low  persisters  fc  =  .47,  p 
=  003). 

Discussion 

The  purpose  of  this  study  was  to  examine  the  effects  of  persistence  on  students' 
ability  to  interact  and  learn  in  cooperative  learning  groups.  In  addition,  the  study  assessed 
the  effect  of  collaboration  on  students'  attitudes  toward  their  partners.  Results  indicate 
that  although  persistence  did  not  affect  achievement,  persistence  did  influence  the  amount 
and  nature  of  interaction  in  groups. 

Average  persisters  interacted  more  frequently  than  did  either  high  or  low  persisters. 
One  possible  explanation  for  this  finding  is  that  average  persisters  are  more  able  and 
consequently  less  apprehensive  about  verbal  interaction  than  other  students.  Carrier  and 
Williams  (1988)  found  a  curvilinear  relationship  between  persistence  and  achievement 
indicating  that  average  persisters  are  more  able  than  high  and  low  persisters.  Average 
persisters  apparently  used  their  better  developed  metacognitive  skills  to  judge  the  optimum 
level  of  effort  to  invest.  Higher  metacognitive  ability  may  also  increase  self-efficacy,  allowing 
individuals  to  benefit  from  group  activities.  Students  who  display  characteristics  of  low 
apprehension  "...  have  well  developed  social  skills,  are  outgoing,  enterprising,  original, 
verbally  fluent,  and  fluent  in  thought  They  possess  self-assurance,  and  are  spontaneous, 
expressive,  and  enthusiastic/'  (Bouchard,  1969).  Consequently,  students  with  higher  levels 
of  metacognitive  ability  may  be  more  willing  to  engage  in  group  interaction. 

As  expected,  a  significant  positive  relationship  was  found  between  the  positive 
verbal  interaction  rate  and  achievement.  However,  the  magnitude  of  the  relationship  is 
lower  than  that  observed  during  other  studies  (e.g.  Hooper,  1991;  Webb,  1982c).  Further 
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investigation  revealed  that  the  relationship  between  verbal  interaction  and  achievement 
was  significant  for  low  persistence  students  only.  Even  though  low  persisters  interacted 
significantly  less  than  average  persisters,  the  interaction  rate  was  a  much  better  predictor  of 
success  for  low  persisters  than  for  others. 

The  results  of  the  Partner  Preference  Questionnaire  were  particularly  interesting. 
Students  were  asked  on  two  occasions  to  name  up  to  five  class  members  with  whom  they 
would  like  to  work  in  a  cooperative  group.  The  first  occasion  was  before  students  had 
completed  any  of  the  lesson  materials  and  the  second  was  immediately  before  completing 
the  posttest.  Two  students  who  originally  named  their  partner  did  not  rename  that 
individual  on  the  follow-up  survey.  However,  twenty-three  students  who  did  not  originally 
name  their  partners  did  so  following  the  group  activity.  In  other  words,  an  individual  was 
significantly  more  likely  to  be  named  as  a  desirable  partner  after  collaborating.  This  results 
support  the  notion  that  collaboration  breaks  down  social  barriers  and  helps  to  improve 
interpersonal  relations  among  students.  Similar  results  have  been  observed  in  many  other 
cooperative  learning  studies.  For  example,  Sharan  and  his  associates  (Sharan,  1980; 
Sharan,  Kussell,  Sharan,  &  Bejarano,  1984)  have  reporter  that  cooperative  learning 
improves  students'  self  esteem  and  their  attitudes  toward  peers  and  school  work.  Similarly, 
Johnson  and  Johnson  (1989)  indicated  that  promotive  interaction  among  partners  during 
cooperative  learning  produces  a  positive  psychological  climate  and  increases  attraction 
toward  one  another. 

Many  teachers  use  cooperative  learning  to  improve  students'  attitudes  toward  each 
other.  However,  the  result  of  the  present  study  is  noteworthy  because  of  the  limited 
exposure  to  partners  experienced  by  participants.  Students  worked  with  their  partners  on 
two  occasions,  generally  for  less  than  one  hour  in  total.  Apparently,  even  limited  exposure  to 
other  students  in  a  meaningful  learning  environment  has  the  potential  to  enhance  students 
attitudes  toward  each  other. 

Further  research  is  needed  to  examine  how  students  attitudes  toward  their  partners 
evolve.  One  common  approach  employed  by  teachers  involves  changing  group  members 
frequently  to  avoid  potential  conflict  between  members.  Yet,  in  practice,  groups  may  become 
more  cohesive  and  effective  with  experience.  Charrier  (1972)  outlined  a  model  of  group 
development  through  which  effective  groups  pass.  Groups  typically  move  through  five  stages 
of  development:  forming,  storming,  norming,  performing,  and  adjourning.  Storming  is 
perhaps  the  most  significant  evolutionary  stage,  because  by  expressing  and  working  through 
this  stage  of  intense  conflict,  group  members  begin  to  assume  the  responsibility  of  managing 
the  intra-group  functions  and  dynamics  (McClure,  Miller,  &  Russo,  1992). 

In  the  present  study  many  partners  apparently  warmed-up  to  each  other  during  the 
second  day  and  became  very  active  in  their  verbal  elaboration  of  the  content.  For  example, 
one  group  that  began  negatively  appeared  to  work  through  their  differences.  One  student 
dominated  the  mouse  at  the  beginning  of  the  first  lesson  and  refused  to  select  additional 
examples  despite  her  partner's  protest.  However,  the  more  persistent  partner  continued  to 
seek  elaborative  information  and  the  previously  negative  partner  appeared  to  develop 
greater  interest  in  the  lesson  content. 

Furthermore,  students  attitudes  toward  their  partners  are  likely  to  reflect  the 
quality  of  intra-group  interaction.  Nine  of  the  interaction  rating  forms  included  comments 
from  raters  concerning  how  well  the  members  were  elaborating  on  the  lesson  content.  These 
nine  groups  contributed  six  of  the  twenty-three  posttest  partner  preference  conversions 
discussed  previously. 

Not  all  interactions  within  groups  were  positive.  Despite  being  given  explicit 
instructions  and  practice  in  how  to  interact  in  groups,  some  partners  did  not  interact 
effectively.  Indeed,  some  students  were  openly  hostile  toward  their  partners  on  both  days. 
For  example,  on  the  first  day  one  group  member  only  pointed  at  the  computer  screen  and  did 
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not  interact  verbally.  On  the  following  day,  the  silent  member  did  not  consult  his  partner 
and  said  loudly  "Don't"  when  his  partner  tried  to  use  the  keyboard  and  then  dominated  the 
mouse  and  keyboard  during  the  rest  of  the  lesson.  Students  who  fail  to  interact  effectively 
are  less  likely  to  reap  the  potential  affective  benefits  of  cooperative  learning. 

Unexpectedly,  the  study  also  produced  a  gender  effect  for  achievement.  Males  scored 
significantly  higher  on  both  posttests  than  did  females.  One  possible  explanation  for  this 
finding  concerns  the  males'  tendency  to  dominate  the  learning  environment.  Informal 
observations  of  the  videotapes  indicated  that  males  tended  to  monopolize  the  computer  and 
keyboard.  For  example,  in  one  case,  objections  from  a  female  were  ignored  by  the  male  until 
the  lesson  was  virtually  complete.  If  these  limited  observations  reflect  a  practice  that 
occurred  throughout  the  study,  then  females  may  have  been  alienated  toward  the  lesson 
resulting  in  poor  encoding  of  lesson  content  and  poor  posttest  achievement. 

Further  research  is  needed  to  analyze  intra- group  interaction.  Dynamic  group 
interaction  is  difficult  to  measure  accurately.  In  the  present  study,  video  tapes  of  the  group 
sessions  provided  an  opportunity  for  in  depth  observations.  In  future,  however,  research 
approaches  that  emphasize  qualitative  methodologies  may  be  particularly  helpful. 

Further  research  is  needed  to  test  the  hypothesis  that  grouping  negated  the  effects  of 
persistence.  Although  the  initial  persistence  scores  of  students  working  alone  appear  to  be 
very  similar  to  those  obtained  in  the  oiiginal  study,  achievement  scores  failed  to  replicate 
the  curvilinear  relationship  between  persistence  and  achievement  found  by  Carrier  and 
Williams  (1988).  Furthermore,  the  type  and  frequency  of  interaction  among  different  levels 
of  persisters  warrants  further  investigation. 
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Table  1.  Means  and  Standard  Deviations  by  Persistence  Level 


Interaction  Concept  Problem 

Rate  Learning  Solving 


Persistence  Level 

High 

Heterogeneous       M  2.68  13.64  2.08 

SD  1.28  7.23  1,38 

N  25  25  25 

Homogeneous        M  2.55  15.89  2.57 

SD  1.09  6.65  1.40 

N  26  28  28 

Total                   M  2.62  14.83  2.34 

SD  1.18  6.95  1.40 

N  51  53  53 

Average 

Heterogeneous       M  3.26  15.04  2.19 

SD  1.48  6.39  1.30 

N  26  26  26 

Homogeneous        M  3.43  12.13  1.88 

SD  1.35  6.55  1.63 

N  16  15  16 

Total                   M  3.33  13.98  2.07 

SD  1.41  6.52  1.42 

N  42  41  42 

Loyr 

Heterogeneous       M  2.67  15.44  2.22 

SD  1.13  6.00  1.54 

N  23  23  23 

Homogeneous        M  2.62  13.81  1.75 

SD  1.26  7.38  1.48 

N                      16  16  16 

Total                   M  2.65  14.77  2.03 

SD  1.17  6.56  1.51 

N                       39  39  39 
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INTRODUCTION 


Purpose 

This  study  was  designed  to  investigate  the  effects  of  the  presence  or  absence  of 
metacognitive  skill  tools  available  in  the  hyperspace  on  both  field  independent  and 
field  dependent  learners.  The  learners  were  engaged  in  problem  solving  in  an 
information  rich  hyperspace  based  on  a  lesson  of  the  attack  on  Pearl  Harbor.  They 
were  given  the  Group  Embedded  Figures  Test  (GEFT)  in  order  to  identify  those  who 
were  predominately  field  independent  and  those  who  were  predominately  field 
dependent.  The  information  acquisition  paths  that  the  learners  took  through  the 
problem  solving  were  automatically  recorded  and  investigated.  The  students' 
solutions  to  the  problem  solving  task  were  analyzed  to  determine  the  recall  of  the 
content  and  the  level  of  the  cognitive  complexity  based  on  the  quality  and  the 
processes  used  to  reach  the  conclusions.  In  addition,  the  amount  of  time  spent  on 
the  key  information  (depth  of  information  processing)  was  recorded  within  the  audit 
trails  so  that  the  information  processing  time  by  different  learners  was  identified  as 
well.  The  significance  of  the  research  findings  for  instructional  courseware  designers 
as  well  as  implications  for  future  study  is  discussed. 

Theoretical  Framework 

Metacognition  and  Learning 
The  theory  of  metacognition  is  concerned  with  the  individual's  awareness, 
planning  and  control  over  cognitive  process  and  learning  activities  (Swanson,  1990; 
Haller,  Child  &  Walberg,  1988;  FlavelJ.979).  Metacognitive  strategies  such  as 
teaching  learners  knowledge  about  what  reading  strategies  to  use  and  how  and 
when/why  to  use  them  produce  reading  comprehension  (Haller,  Child  &  Walberg, 
1988).  In  addition,  studies  indicated  that  subjects  required  to  explain  reasons  for 
actions  while  solving  problems  demonstrated  a  positive  transfer  of  tasks 
(Stinessen,1985).  Lin  (1993)  compared  metacognitive  groups  with  cognitive, 
affective,  and  control  groups.  She  found  that  metacognition  had  a  significant  positive 
impact  on  students'  transfer  of  the  problem  solving  tasks.  It  was  suggested  that 
metacognitive  processes  made  students  focus  on  and  become  consciously  aware  of 
the  solution  components.  Because  of  this,  learners  were  able  to  attend  to 
information  regarding  the  way  to  solve  a  problem.  Unfortunately,  it  appears  that 
metacognitive  processes  do  not  take  place  spontaneously.  It  requires  the 
development  of  awareness  of  task,  strategy,  and  performance  (Wade  & 
Reynolds,  1989).  Learners  may  also  gain  metacognitive  skills  through  the  teacher's 
modeling  or  guided  practice  (Billingsley  &  Wildman,  1990),  prompted  cues  (Lin, 
1993  ),  or  self-questioning  (Wade  &  Reynolds,  1989).  However,  the  existing 
instructional  design  models  do  not  typically  emphasize  metacognitive  strategies  such 
as  planning,  monitoring,  revising,  and  other  self-regulating  activities.  (Osaman  et. 
al.  ,1992). 

Field  dependence/field  independence 
Cognitive  style  refers  to  an  individual's  characteristic  approach(es)  to  perceiving, 
processing,  and  organizing  information.  It  was  proposed  that  the  construct  field- 
dependent/independent  is  one  of  the  cognitive  styles  most  significant  to  education 
problems  and  has  been  more  extensively  researched  (Chinnien  &  Boutin,  1993). 
Research  indicated  that  the  field  independent  style  is  mostly  related  to  the  ability 
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to  perceive  a  particular  relevant  item  in  a  "field"  of  distracting  items.  Conversely, 
field  dependent  people  tend  to  be  dependent  on  the  total  field  and  hence  the  parts 
embedded  within  the  field  are  not  easily  perceived  (Brown,  1987).  As  a  result, 
compared  to  field  independent  learners,  field-dependent  individuals  are  more  likely 
to  have  greater  difficulty  in  learning  when  the  learner  himself/herself  is  required  to 
provic1^  organization  as  an  aid  to  learning.  (Wiktin,  Moore,  Goodenough  &  Cox,  1977 
).  Da .  is  (1989a)  added  in  a  review  of  studies  on  field-dependence  that  the  field- 
dependent  learner  is  more  reliant  on  salient  cues  in  learning.  Davit*  further 
suggested  incorporating  metacognitive  skills  into  future  learning  research  of  field 
dependence/independence  (Davis,  1989h). 

Navigational  path*  of  information  in  hvpeispai  e 
Hypermedia  systems  integrate  text.  sound,  tunnlm  h,  and  video  lAvi  jsnmn, 
1993;  Wang  &  Jonassen,  1993)  to  present  the  knowledge  wMhin  the  network  of 
ideas  or  use  nodes  and  links  to  orgain/e  stun  i tit ■■  tKjilin  K  JehtiH,  ItHNI)  The 
emergence  of  hypermedia  ha*  fostered  the  rotopntet  hawed  learning  a  iinieh  |>eer  mid 
richer  environment  which  provide*  freetlntn  tn  the  iiwi  In  evpl'iir  (Kiatnit.  A  Itaihei , 
1992).  Unlike  traditional  learning  niivn onnti  ut .  it  apj^aiw  that  hypu  media 
learning  is  self-direrted.  Withtht^  s\Mein  (he  i «>h|h tiihiliihl \  toi  identifying  wlmt  is 
useful  information  and  selecting  m  at  %  Ii  rdutepn  t.  Ini  in  <  i  "*inn  (hat  inhumation  ik 
largely  left  up  to  the  oner  (Small  K  i U  aim*  ski  I  W!M    I  In  irhae  (he  ledum  or  the 
user  is  required  to  internet  ami  exploie  (In  infra  mafnai  I » v  ih  v eloping  their  own 
paths  or  knowledge  strurtine 

Due  to  the  user-orient ed  natnn  ul  h\ | >«  um  ilei  (In  h-.iiiii-iM  in  eit  In  deride  where 
to  retrieve  information,  what  h( i  ntijiti  p>  to  om  Im  pinlih m  m»Is  mg  mill  why  oi  when 
to  quit  the  environment    It  is  thus  In  lit  vt  d  \Uii\  on  tin  ngnitue  h»ow ledge  ami  skills 
are  essential  for  suereRsful  in  hvpeimedhi  r\-  Inn  ll  Jii  hn  atlHe  in  the 

metacognitive  process  the  learnein  nreil  te,  plun  lm  »  ninnlive  at  Imiis,  monitor  their 
progress,  and  refleet  upon  ami  regulate  then  <iwu  l«  lit n imm  pim  run  (Howe,  lilHH; 
Cardelle-Klawar,  1992)  While  it  won  ptmed  that  inelaiognltioli  played  an 
important  role  in  oral  comprehension,  lending  i  empti  Ih'IimihIi.  piohlem  solving, 
memory  and  the  like  (Mavel.1979),  the  hm  uii  Ii  limhiigii  ah~n  showed  that 
incorporated  metacognitive  strategies  in  h\ pel  media  learning  would  eiihaiiee 
problem  solving  and  transfer  of  tasks  <l,in.l!MW) 

The  literature  has  shown  that  tin*  hypei  methu  s\stein  eliimnateH  linear  linkage 
of  text  and  allows  users  to  freely  browse  thiotigh  n  knowledge  huso  (Nelson  & 
Palumbo,  1992).  Yet,  it  has  been  observed  thn(  lent  nets'  motivation  may  h« 
impaired  if  they  become  overwhelmed  by  the  freedom  in  (he  hypermedia  environment 
(Staton  &  Barber,1992).  The  recent  development  of  hypermedia  has  henc  eforth  put 
more  concern  on  issues  of  navigational  problems  and  the  sea  nil  strategies  by 
different  individuals.  Jonassen  (1988)  advocated  that  it  is  important  to  investigate 
how  learners  navigate  through  hypermedia  systems  and  how  individual  differences 
could  predict  those  paths. 

In  concert  with  the  above  concerns,  the  present  study  demonstrates  the 
significance  of  studying  how  field  dependent  and  independent  learners  interact  with 
the  hypermedia  system.  It  is  especially  important  to  explain  whether  metacognition 
plays  an  influential  role  for  individual  differences  in  cognitive  style  if  the  individual  is 
cued  to  consider,  as  well  as,  evaluate  his  or  her  thoughts  by  an  available  tool  to 
process  metacognition  during  the  problem  solving  process  in  a  hyperspace 
environment. 
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Statement  of  research  problems 
This  research  study  aimed  to  investigate  how  field  dependent  and  field 
independent  people  respond  to  the  presence  or  absence  of  metacognitive  skill  tools  in 
information-rich  hyperspace.  The  hyperspace  is  defined  here  as  loosely  structured 
hypermedia  where  text,  charts,  document,  and  graphics  wore  integrated  and 
hyperlinked  (Chung,  Frederick,  Hsu,  1994).  In  the  hyperspace  environment,  the 
expert  structure  was  not  created  by  the  developers  or  programmers.  The  subjects 
were  expected  to  bring  their  own  expert  systems  of  structure*  to  solve  the  problem. 
This  research  investigated  the  following  research  problems 

1.  Will  the  presence  of  metacognitive  skill  tool*  have  on  effeet  on  field 
dependent  or  field  independent  learner*'  recall  n!' content  ami  level  of 
thinking  process  while  providing  solution*  to  the  pifthlems'' 

2.  Will  the  presence  of  metacognitive  skill  tool*  have  an  etf< <t  on  field 
dependent  or  field  independent  learners'  depth  ol  iftifoi  mat  ion  processing? 

3.  Will  the  presence  of  metacognitive  nktll  t<ml*  have  an  rllin  t  nn  field 
dependent  or  field  independent  learners  ittfoMtnttinn  seeking  paths'* 

SigntfiLaawv  s&  Ibc  i'lui^kui 

The  infusion  of  computer-based  intermtiie  li\pe»  media  mvo  mMiui  tioiml 
environments  has  made  a  dramatic  change  in  edih  at mn  Sim  e  then,  the 
professionals  have  been  investigating  how  intelaihve  Inpeniiedia  materials  might 
be  designed  so  that  they  are  developitienttillv  appiopmite  ilnnassen  tlUHH) 
proposed  his  insight  into  maximizing  the  hypeitext  learning   He  hmiiI  that  a  path 
analysis  program  needed  to  he  developed  to  identity  am  phimliient  access  path 
learners  take  through  hypermedia  environments  He  also  suggested  relating  the 
access  paths  to  different  learning  styles.  The  advocation  by  ♦haiassen  serves  the 
basis  of  rationale  for  this  study. 

Aust  and  Klayder  (1090)  have  a  eoncei  n  about  an  overabundance  of  information 
and  predict  that  it  may  result  »n  "information  anxiety  or  knowledge  starvation".  In 
fact,  often  learners  have  a  tendency  to  become  lost  or  disoriented  while  using 
hypermedia  environments.  Specifically,  it  was  found  that  ''individuals  often 
encounter  loosely-structured  information  environments  wherein  there  are  no  clear 
questions  to  be  answered"  (('hang  &  Mc Daniel  1993).  It  is  suspected  that  this 
problem  could  be  related  to  a  variety  of  learner  characteristics.  It  is  possible  that 
the  "learners  lost  in  hyperspace*'  may  be  predominantly  field  dependent  learners 
since  field  dependent  people  have  difficulty  with  providing  organization  to  learning 
(Wiktiu  et.  al.,  1977).  In  particular,  it  is  predicted  by  the  researchers  that  field 
dependent  people  may  become  efficient  learners  if  metacognitive  tools  are  provided  in 
the  hyperspace  environment.  Based  on  the  research  assumptions,  it  is  important  to 
prove*  the  hypothesis  to  determine  if  the  integration  of  metacognitive  skill  tools  is  a 
viable  method  for  developing  a  better  hyperspace  environment  for  different  learners. 
Moreover,  since  hyperspace  is  a  loosely-organized  hypermedia,  the  students  are  not 
provided  with  "expert  structure"  to  the  system.  Conversely,  the  students  need  to 
develop  their  own  structure  of  navigating,  seeking  and  processing  information  in 
hyperspace.  The  findings  of  this  study  would  have  significant  contribution  to 
determine  what  an  expert's  structure  is  and  thus  help  to  develop  a  hypermedia 
learning  environment. 

METHODOLOGY 
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Subjects 

Forty  undergraduate  students  who  were  registered  in  an  introductory  computing 
course  at  a  midwestern  university  participated  in  this  study.  The  students 
automatically  earned  credits  toward  their  course  grade  for  participation.  There 
were  two  males  among  the  subjects.   There  was  one  subject  whose  ethnic  origin  was 
Asian  and  the  rest  were  Caucasians.  The  computer  experience  reported  by  the 
subjects  varied  from  fair  to  excellence.  The  familiarity  of  the  research  content,  M  arl 
Harbor,  ranged  from  fair  to  average. 

Material  I 

In  this  study,  a  lesson  called  "Asleep  at  Dawn-The  Attack  on  Pearl  Harbor"  was 
constructed  on  a  Macintosh  HyperCard  stack  by  one  of  the  researchers.  This  lesson 
provides  a  wealth  of  information  about  the  Pearl  Harbor  attack.  The  content 
materials  were  based  on  the  following  books:  Pearl  Harbor  Final  Judgment 
(Clausen,  H.  &  Lee  B.,  1992),  America  Goes  to  War  1941  (Devaney,  J.,  1991),  And  I 
Was  There  (Layton,  E.  at  el  1985),  Pearl  Harbor  (Shapiro,  W.  1984),  and  79th 
Congress  of  United  States  of  America  (1946). 

The  materials  were  presented  with  the  following  distinctive  features.  First,  the 
subjects  were  provided  the  introduction  to  the  lesson.  They  were  requested  to  play  a 
role  as  an  investigator  to  investigate  the  events  surrounding  the  bombing  of  Pearl 
Harbor.  Second,  the  main  body  of  information  about  the  Pearl  Harbor  attack  was 
presented  via  hyperlinks  (Append  :jt  I).  There  are  twenty  topics  that  can  be  chosen 
from  the  main  menu.  Access  to  information  chosen  by  the  individual  reflects  his  or 
her  own  paths  and  speed  of  searching  for  information  among  the  topics.  The 
subjects  could  quit  whenever  they  felt  they  had  enough  information  for  the 
conclusions  of  their  investigation.  Third,  there  is  a  menu  topic  called  "Four  Key 
Questions"  embedded  in  the  main  menu.  The  "Four  Key  Questions'1  comprises  four 
questions  which  serve  as  guided  clues  to  stimulate  the  subjects'  critical  thinking.  In 
addition,  a  menu  called  "Mission"  was  created  and  placed  on  the  menu  bar.  The 
"Mission"  menu  describes  the  tasks  that  the  subjects  need  to  accomplish  throughout 
the  Pearl  Harbor  lesson.  Fourth,  there  are  some  cards  with  three  asterisks  labeled 
to  denote  that  these  cards  contain  key  information  for  answering  questions  in  the 
conclusions.  Fifth,  there  are  two  questions  posed  in  the  conclusions  where  the 
subjects  needed  to  determine  the  critical  events  that  caused  the  attack  and  whom 
should  be  blamed.   When  the  subjects  clicked  "Done"  on  the  menu,  the  conclusion 
screen  was  provided  and  the  subjects  needed  to  supply  their  answer.  This  stack 
also  includes  a  path  tracking  mechanism  which  records  simultaneously  the  paths 
chosen  by  the  subjects  and  time  used  by  the  subject  for  each  level  of  information 
access. 

In  this  study,  even  though  either  the  "Four  Key  Questions"  or  "Mission" 
function  as  guidance  which  will  help  the  subjects  form  an  expert's  knowledge 
structure  in  the  hyperspace,  the  subjects  were  not  particularly  informed  about  why 
and  when  to  get  access  to  this  information. 

Material  II 

This  lesson  has  everything  covered  in  Material  I  except  that  Material  II  has 
metacognitive  tools  available  on  the  menu  bar  called  META.  The  META  tools  posed 
four  different  questions  at  one  time.  For  each  selected  question,  the  subjects  were 
prompted  to  enter  their  plans  for  the  investigation  and  their  reason  for  this 
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particular  plan  (Appendix  II).  The  plan  and  reason  are  not  the  answer  to  the 
question  being  posed  but  rather  the  students'  current  thought  and  actions  taken  to 
accomplish  the  investigating  task.  In  other  words,  the  questions  were  intended  to 
help  the  subjects  structure  their  paths  of  searching  the  information.  The  subjects 
had  to  check  the  META  menu  periodically  until  all  four  questions  were  selected. 
Unlike  Material  I,  subjects  using  Material  II  were  required  to  use  the  expert 
questions  and  to  enter  their  metacognitive  processes  in  terms  of  their  plans  and 
reasons  for  the  plans. 

Data  Collection 

Independent  Measures 
The  Groups  Embedded  Figures  Test:  The  Group  Embedded  Figures  Test  was 
administered  before  the  treatment  to  measure  the  learners'  cognitive  styles.  The 
GEFT  has  been  used  widely  as  a  valid  and  reliable  instrument  to  assess  the 
cognitive  style  (Small  &  Grabowski,  1992). 

Treatment  Conditions:  The  subjects  were  randomly  assigned  to  different  treatment 
conditions  during  the  hyperspace  problem  solving  session. 

1.  Field  dependents  with  metacognitive  tools  available  (Experimental  group). 

2.  Field  independents  with  metacognitive  tools  available  (Experimental 
group). 

3.  Field  dependents  without  access  to  metacognitive  tools  (Control  group). 

4.  Field  independents  without  access  to  metacognitive  tools  (Control  group). 

Dependent  Measures 

This  study  attempted  to  investigate  the  effects  of  the  presence  or  absence  of 
metacognitive  tools  in  the  hyperspace  environment  on  field  dependents*  and  field 
independents'  recall  of  content,  level  of  cognitive  process,  depth  of  information 
process,  and  information  seeking  strategies. 

L  Recall  of  content:  The  conclusions  provided  by  the  subjects  were  evaluated 
from  two  aspects:  recall  of  content  and  level  of  cognitive  process.  A  checklist  of  the 
critical  events  which  caused  the  Pearl  Harbor  attack,  or  key  characters  who  should 
be  responsible  for  the  surprise  attack  was  developed  by  the  stack  developer  as  the 
scoring  criteria  for  the  recall  of  the  content. 

2.  Cognitive  complexity:  The  approach  to  scoring  subjects'  cognitive  complexity 
on  conclusion  was  to  read  them  holistically  and  examine  the  reasons  supporting 
their  position.  The  conclusions  were  rated  on  a  scale  of  one  with  simple  description 
of  the  fact  to  five  with  integrated  analysis  of  learning  content.  This  rating  scale  was 
used  to  determine  the  cognitive  complexity  which  was  developed  by  McDaniel  and 
Lawrence  (McDaniel  &  Lawrence,  1990).  Due  to  the  subjective  nature  of  essay 
tests,  two  independent  scorers  were  involved  to  gain  interrater  reliability  in  scoring. 

3.  Depth  of  information  processing  was  determined  by  time  spent  on  each  key 
information  denoted  by  three  asterisks.  The  key  information  was  considered 
providing  useful  content  for  drawing  conclusions  of  investigation. 

4.  Information  Seeking  Strategies  were  identified  by  the  mean  time  learners 
spent  on  searching  for  a  topic,  the  subjects'  frequent  use  of  the  "Mission"  menu  and 
"Four  Key  Questions"  as  monitoring  guidance,  and  descriptive  data  of  the  subjects' 
audit  trails  to  reveal  individual's  navigational  patterns.  Qualitative  analysis  of  the 
subjects'  exit  questionnaire  was  conducted  as  a  support  of  the  empirical  data. 
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Pilot  Study 

Before  the  actual  experiment,  a  pilot  Htuily  wiih  administered  to  confirm  that  no 
problems  existed  in  the  HyperCard  program  or  in  the  exit  questionnaire.  A  student 
with  similar  background  of  the  anticipated  subjects  volunteered  to  participate. 
Some  minor  changes  were  made  according  to  the  results  of  the  pilot  study. 

Procedures 

The  whole  experiment  is  divided  into  two  sessions  Held  on  different  days.  The 
first  session  was  scheduled  for  assessing  the  students'  cognitive  styles.  The  whole 
process  of  assessment  was  timed  based  on  the  itiHtrurtioiiH  in  the  manual.  Based 
on  the  scoring  result,  forty  subjects  (field  independency  =2(1;  field  dependency^  20) 
were  selected  and  randomly  assigned  to  one  of  four  different  experimental  conditions. 
The  experiment  proceeded  in  the  same  locution  hut  different  rooms  for  control  and 
experimental  groups  respectively,  Before*  beginning  the  lesson,  all  students  were 
given  a  group  orientation  to  the  lesson  by  the  researchers.  The  subjects  were 
briefed  on  the  use  of  buttons  in  hyperciird  and  their  mission  throughout  the  lesson. 
People  in  the  experimental  group  were1  also  told  tn  n*e  the  metacognitive  tools 
available  on  the  menu  bar.  Once  instructed  to  slill't,  the  subjects  in  both  groups 
worked  individually  in  separate  rooms.  The  whole  computer  lesson  lasted 
approximately  from  one  tu  one  and  half  hours.  Upon  completion  of  the  lesson,  the 
subjects  filled  out  an  exit,  quest  ion  not  re 


Data  Analysis  Tools 
The  subjects1  audit  trails  of  working  with  the  hypet  space  were  collected  and 
analyzed  by  a  statistical  software*  package  called  Stat  View  SK+.  ANOVA  tables 
were  obtained  to  determine  the  main  effects  and  the  interaction  between  the  two 
independent  variables  on  Hut  dependent  variables,  In  addition,  a  thorough 
examination  of  the  subject*  paths  and  self-reported  searching  strategies  were 
analyzed  through  a  qualitative  approach. 

RESULTS 

Recall  of  the  Content 
The  ANOVA  table  concerning  students'  recall  of  content  are  presented  in  Table  1. 
The  mean  scores  between  the  groups  that  received  the  metacognitive  tools  and  those 
who  do  not  have  the  access  are  not  significantly  different  (F=1.054,  p=.3107). 
Students  performed  similarly  on  the  recall  of  content  regardless  of  the  availability  of 
the  metacognitive  tools  in  the  hypermedia  learning,  however,  a  significant  main 
effect  was  found  for  the  cognitive  styles  (F=6.475,  p=.015).  The  field  independent 
people  outperformed  the  field  dependent  people  in  recalling  the  content.  No 
interaction  between  treatments  and  cognitive  styles  was  found. 

Cognitive  Complexity 
A  significant  main  effect  was  found  for  the  treatments  (Table  II).  Students  who 
are  not  provided  with  metacognitive  tools  demonstrate  a  higher  level  of  thinking  in 
drawing  conclusions  than  those  who  have  access  to  the  metacognitive  tools  (F=4,52a, 
P=.0397).  The  mean  scores  of  the  field  independents  are  found  to  be  significantly 
higher  than  the  field  dependents  (F=8.551,  P=.0O57).  No  interaction  exists  in  this 
particular  dependent  variable. 
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Depth  of  information  processing 
There  is  no  main  effect  found  in  the  depth  of  information  processing  (Table  III). 
Students  spent  approximately  the  same  amount  of  time  processing  the  key  or  critical 
information  regardless  of  the  cognitive  styles  or  treatments  that  students  were 
receiving.  It  was  found,  however,  that  there  is  correlation  existing  between  the  recall 
of  content  and  the  number  of  cards  that  had  been  read  (r=.265,  p<.05)(Table  IV  a). 
The  students'  level  of  cognitive  complex  shown  in  the  conclusions  is  also  correlated 
with  totul  cards  that  were  read  (r=  294,  p<,05)  (Table  IVb.), 

Information  Seeking  Strategies 
The  searching  strategics  were  studied  from  three  perspectives:  time  on  searching 
(how  much  time  wan  spent  on  deciding  on  a  topic)*  frequent  use  of  monitoring  or 
guidance  (how  often  did  the  students  chec  k  the  "Mission"  menu  or  "Four  Key 
(Juestions")  mid  ntivigutiomil  patterns  (in  what  order  the  topic  was  accessed).  No 
statistical  iniiin  efforts  were  found  between  the  groups  in  spending  time  in  deciding 
a  topic  fmm  n  variety  uf  options  on  the  main  menu  (Table  V  ). 

Kvon  though  no  significant,  result  was  indicated  on  the  frequent  use  of  monitoring 
between  the  groups,  ttiere  was  a  moderate  difference  in  the  use  of  monitoring  by  field 
independents  and  dependents  (l'=.(Ml,  p<.l)(Table  VI).  The  field  dependent  people 
tended  to  constantly  check  their  instructional  objectives  (Mission)  and  using 
guidance  (Kour  Key  Questions)  more  often  than  did  the  field  independents 
throughout  the  hyperspace  problem  solving  session. 

As  the  subjects'  audit  trails  were  analyzed,  a  detailed  picture  of  the  navigational 
patterns  emerges.  There  were  three  most  prevalent  navigational  patterns:  1. 
Linear  access:  The  subjects  chose  the  topics  on  the  main  menu  in  the  order  of  top  to 
bottom  or  left  to  right  (as  defined  by  Small  et.  al.,  1992;  Chung  et.  al.,  1994)  2. 
Non-linear  access:  The  subjects  chose  the  topics  without  any  distinctive  structured 
manners.  The  orders  were  jumped  around.  3.  Cycle  access:  The  navigation  in  the 
hypermedia  falls  into  a  cycle  pattern.  The  subjects  kept  going  back  and  forth  among 
the  chosen  topics.  The  navigational  patterns  only  reveal  the  individual  preferences 
of  information  access.  The  finding  did  not  indicate  that  particular  patterns  emerged 
due  to  differences  in  cognitive  styles  or  the  manipulation  of  the  treatments.  It  is 
worthy  of  notice,  however,  that  five  out  of  the  top  eight  people  who  scored  both  high 
on  recall  of  content  and  cognitive  complexity  tended  to  navigate  linearly  in  the 
hyperspace. 

Exit  Questionnaire 
Self  reported  searching  strategies:  In  addition  to  the  quantitative  data, 
data  gathered  from  the  questionnaire  were  analyzed  to  provide  an  in-depth 
and  valid  interpretation  of  the  students'  information  searching  strategies. 
The  major  assertion  generated  from  the  students'  report  was  that  they 
selected  the  paths  based  on  the  condition  that  the  information  was  vital  to 
provide  valuable  knowledge  for  their  conclusions. 

7  chose  what  seemed  to  be  the  next  important  information. " 

"I  asked  myself  what  kind  of  information  I  am  going  to  find,  how  this 

information  would  help  me  in  finding  out  more  about  the  attack. " 

7  chose  the  next  path  if  it  seemed  like  the  information  would  be 

helpful  for  me  in  solving  this  problem" 


Another  common  strategy  employed  by  the  students  was  that  they  chose  the 
paths  which  contained  information  pertinent  to  what  they  had  previously  known. 
"(I  chose  the  path  based  on)  what  I  have  previously  found  out'9 
"It  seemed  important  to  choose  the  next  item  that  was  somewhat 
related  to  the  one  I  just  looked  at" 

The  use  of  the  metacognitive  process  demonstrate  an  indicator  of  I  he  n(  orient  n* 
strategies  for  searching  information.  The  students'  plans  and  explanation  lor 
planning  revealed  what  is  the  next  path  she  or  he  is  going  to  take. 

"I plan  to  find  out  what  Pearl  Harbor  knew  about  what  the 
Japanese  were  planning  on  doing.  I  want  to  do  it  this  way  ho  I 
can  try  to  figure  out  why  they  withheld  information  about  the 
intentions  of  the  Japanese. " 

"/  want  to  investigate  on  those  individual  leaders  who  seem  to 
have  some  connection  with  the  Japanese.  If  there  was  anyone  in 
Washington  that  knew  anything  about  the  plans.  I  have  to 
investigate  all  of  the  main  people  in  charge  of  US  operations,  Hy 
looking  indepthly  at  these  various  leaders,  hopefully  I  can  gain 
some  clues  on  where  information  on  Japanese  plans" 

In  addition,  there  were  some  students  who  employed  the  guided  questions  to 
direct  them  to  the  next  path. 

"(I  chose  the  next  path  for)  gaining  information  that  would  provide 
answers  to  the  questions." 

"At  first  I  wanted  to  find  the  important  people.  Then  I  just  clicked 
around  to  find  the  answer  to  the  key  questions" 

As  expected,  some  people  got  lost  in  searching  the  information.  These  people  did 
not  demonstrate  any  strategies  in  their  report. 

"/  was  pretty  random  in  my  selection.  Once  again,  I  didn't  have 
enough  background  information  to  have  a  real  direction." 
"Whether  or  not  to  go  on" 
"How  to  get  back  to  the  main  menu. " 

Attitude  toward  metacognitive  use:  Descriptive  data  were  obtained  with  regard  to 
students1  attitude  toward  the  use  of  metacognitive  tools  in  hypermedia.  The 
assertion  drawn  from  the  analysis  was  that  the  students  conceded  that  planning 
help  them  structure  knowledge  about  the  conclusions,  yet,  they  found  they  were  still 
not  s  .  Uful  at  using  their  "working  mind/1  Some  comments  related  to  this  assertion 
are: 

"Yes,  I  think  that  meta  is  people's  working  mind.  It  helped  me  form  my 
ideas  to  write  the  conclusion,  because  it  helped  me  formulate  the 
questions  that  I  wanted  to  ask  at  the  end,  and  it  also  helped  me  write  on 
the  topic  given." 

"I  do  think  it  (metacognition)  offered  me  some  hints  to  get  conclusion. 
However  as  far  as  the  case  "Pearl  Harbor"  was  concerned,  it 's  hard  for  me 
to  make  initial  plans  without  a  clear  introduction.  That  is  to  say  I  need 
more  knowledge  about  the  background  information.  " 
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"/  thought  it  (metacognition)  was  confusing,  but  Jam  not  used  to 
working  on  a  computer,  so  it  is  all  kind  of  strange.  I  think  planning  out 
your  thoughts  is  very  important,  but  lam  just  not  familiar  with  the 
material  or  the  computer" 

DISCUSSIONS 

Research  findings  in  this  study  indicated  that  the  use  of  metacognitive  tools 
did  not  contribute  to  the  students'  better  learning  with  hyperspace.  On  the  contrary, 
when  the  students'  cognitive  complexity  was  evaluated  based  on  their  conclusions, 
the  control  groups  outperformed  the  experimental  group.  In  other  words,  the 
students  did  not  benefit  from  planning  or  explaining  their  thinking  process  during 
the  problem  solving  tasks.  Through  analyzing  the  students'  protocols^ne  thing 
remains  evident.  It  appears  that  the  lack  of  prior  knowledge  about  the  lesson  plays 
a  vital  role.  Students'  metacognitive  process  were  carefully  screened  by  looking 
through  their  plans  and  explanations.  One  student  wrote  all  the  way  through  in  her 
planning:  "To  get  more  information."  Another  student  expressed  her  opinion  on 
planning, " without  background  or  introduction,  it  is  very  hard  to  form  an  initial 
plan"  When  the  student  is  lacking  background  knowledge,  she  or  he  will  not  have 
the  knowledge  foundation  from  which  actual  plans  can  be  built  up. 

Small  et.  al.  (1992)  advocated  that  the  decisions  the  students  make  are 
influenced  by  one  or  more  personal  or  environmental  constraints,  such  as  prior 
experience,  or  the  type  or  amount  of  information  available.  Their  advocation 
demonstrated  that  the  prior  experience  or  knowledge  brought  by  the  students  has 
influence  on  students'  interaction  with  hypermedia  learning.  The  implication  of  this 
result  in  this  study  is  that  there  in  a  need  for  future  research  to  investigate  how  to 
activate  the  students'  prior  knowledge  or  their  "schemata"  in  the  hypermedia 
learning  environment. 

Another  explanation  for  the  nonsignificant  result  s  of  using  metacognitive  tools  in 
the  hypermedia  could  be  that  the  Ntiuli'titH  nri1  st  ill  not  familiar  with  the  use  of 
metacognitive  strategies  or  an*  not  used  to  employing  the  strategies  in  their  learning. 
While  some  students  felt  that  metacognition  would  help  them  form  their  knowledge 
structure,  other  students  stated  that  they  still  did  not  know  what  metacognition 
was.  This  finding  implies  that  more  time  on  training  student's  metacognitive 
strategies  is  needed.  Lin  (1992)  cites  the  work  of  OeStefano  and  Gordon  (1986)  and 
points  out  that  the  idea  of  training  individuals  to  employ  metacognition  has  been 
stimulated  from  the  findings  that  many  individuals  cannot  independently  produce 
effective  learning  strategies  or  realize  when,  where  and  why  to  apply  known 
strategies,  but  they  can  use  learning  strategies  when  instructed  to  do  so.  The  length 
of  time  on  training  to  achieve  maximum  outcome  is  still  in  need  of  investigation. 

The  findings  do  not  indicate  any  significant  difference  in  the  depth  of  information 
processing  between  the  treatment  groups.  This  result  implies  that  the  learners 
spend  the  same  amount  of  time  on  reading  the  information  which  will  provide 
important  clues  for  investigating  tasks.  The  number  of  curds  that  the  students  went 
through;  however,  was  correlated  with  the  recall  of  the  content  and  higher  level  of 
thinking  process.  In  other  words,  the  more  the  students  spent  time  reading  the 
information,  the  more  the  students  could  recall  and  the  higher  thinking  level  they 
were  able  to  achieve.  Future  research  which  investigates  how  to  motivate  the 
learners  to  better  spend  their  time  in  the  task  could  definitely  improve  effective 
hypermedia  learning. 

The  research  finding  shows  that  neither  cognitive  styles  nor  the  experimental 
treatment  (i.e.  metacognitive  tools)  have  an  effect  on  students'  navigational  paths, 
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Roth  groups  of  people  spent  equal  amount  of  time  searching  for  a  topic.  Yet,  the* 
analysis  of  students'  protocols  show  that  the  use  of  the  metacognitive  process  helped 
the  students  structure  their  searching  strategies.  As  to  the  navigational  patterns,  a 
trend  indicates  that  people  who  are  ranked  in  the  top  on  their  posttest  (i.e.  drawing 
a  conclusion)  tend  to  navigate  linearly  through  the  paths  (i.e.  to  choose  the  topic  in 
the  order  of  top-bottom  or  left-right).   Even  though  the  trend  needs  more  supportive 
empirical  data  or  in  depth  investigation  of  students'  self-reported  description,  it 
appears  that  the  assumption  proposed  by  Spiro  et.  al.(1990)  that  linear 
organization  may  not  permit  or  encourage  the  learners  to  learn  by  exploring  in  their 
full  complexity  requires  further  study  .  Since  hypermedia  are  featured  by  the 
nonlinear  structure,  it  is  suggested  to  investigate  how  to  organize  the  information 
in  order  not  to  discourage  the  students  from  a  non-linear  hypermedia  environment 
especially  when  they  are  required  to  self-control  the  pace  and  paths. 

Even  though  the  field-dependent  people  constantly  monitored  the  instructional 
goal  or  used  guided  questions  relatively  more  often  than  did  the  independent  people, 
the  field  dependent  people  did  not  demonstrate  better  performance  in  drawing 
conclusions.  The  reason  behind  this  finding  is  unknown.  Yet,  Jonassen  &  Wang 
(1992)  found  out  that  the  field  independent  learners  were  better  hypertext 
processors,  especially  as  the  form  of  the  hypertext  became  more  referential  and  less 
overtly  structured.  With  loosely-structured  hypermedia  like  hyperspace  in  this 
study,  Jonassen  et.  al.'s  finding  appeared  to  explain  why  field  dependent  people 
employed  the  monitoring  strategies  yet  they  still  could  not  process  the  information  as 
well  as  the  field  independents.  The  research  findings  by  Weller  et.  al.  (in  press) 
could  support  the  explanation  for  relatively  poor  performance  of  the  field  dependents 
as  well.  Weller  et.  al.  pointed  out  that  "field  independent  students  learned 
computer  ethics  more  effectively  than  did  field  dependent,  even  when  field  dependent 
students  were  provided  with  'help1  such  as  advance  organizers  and/or  graphical 
organizers".  As  the  researchers  voice  their  concern  about  learner  differences  and 
hypermedia  learning  (Ginsburg  &  Zelman,  1988;  Wang  &  Jonassen,  1993),  there  is 
a  large  research  area  left  for  those  who  are  interested  in  hypermedia  learning  and 
learner  characteristics  to  explore  . 

The  use  of  hyperspace  learning  environment  in  this  study  helps  to  shape  the 
instructional  developers1  view  on  how  people  bring  their  own  knowledge  structure 
into  learning.  The  findings  about  the  students1  navigational  strategies,  paths  of 
information  processing  in  the  system  interacting  with  different  cognitive  styles  and 
metacognition  would  be  beneficial  to  the  educators  as  well  as  courseware  designers. 

Table  I:  Effects  on  recall  of  the  content 


ANOVA  table  for  a  2-factor  Analysis  of  Variance  on  Y 


Source 

DF 

Sum  of 
Squares 

Mean 
Squares 

F-test 

P  value 

Treatment  (A) 

1 

12.023 

12.023 

1.054 

0.3107 

Cognitive 
Styles  (B) 

1 

73.841 

73.841 

6.475 

0.015 

AB 

1 

19.114 

19.114 

1.676 

0.203 

Error 

40 

456.182  ' 

11.405 

5:fo 
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The  AB  Incidence  table  on  Y:  Recall  of  content 


Dependent 

Independent      |  Totals: 

Mctji 

n=ll 
mean=4.364 

n=ll 
mean=8.273 

n=22 
mean=6.318 

Non-Meta 

n=ll 
mean=4.636 

n=ll 
mean=5.909 

n=22 
mean=5.273 

Totals: 

n=22 
mean=4.5 

r>=22 
mean=7.091 

n=44 
mean=5.795 

* scores  (0-31)  on  the  conclusions  for  the  recall  ability 


Table  II:  Effcts  on  cognitive  complexity 


Source 

DF 

Sum  of 
Squares 

Mean  Square 

F-test 

P  value 

Treatment 

XA) 

1 

5.818 

5.818 

4.523 

0.04 

cognitive 
Styles  (B) 

1 

11 

11 

8.551 

0.006 

AB 

1 

3.273 

3.273 

2.544 

0.1186 

Error 

40 

'51.455 

1.286 

The  AB  Incidence  table  on  Y:  cognitive  complexity 


Dependent 

Independent      |  Totals: 

Mota 

n=ll 
mean=  1,182 

n=ll 
mean=2.272 

n=22 
mean=l,955 

Non-Meta 

n=ll 
mean=2.455 

n=ll 
mean=2.909 

n=22 
mean=2.682 

Totals: 

n=22 
mean=1.818 

n=22 
mean=2.818 

n=44 
mean=2.3l8 

*scores  (0-5,»  on  the  conclusions  for  cognitive  complexity 


Table  III:  Effects  on  depth  of  information  processing 


ANOVA  table  for  a  2-factor  Analysis  of  Variance  on  Y 


Source 

DF 

Sum  of 
Squares 

Mean  Square 

F-test 

P  value 

Treatment 
(A) 

1 

90.213 

90.213 

0.592 

0.446 

cognitive 
Styles  (B) 

1 

46.25 

46.25 

0.303 

0.585 

AB 

1 

P.227 

9.227 

0.061 

0.807 

Error 

40 

6099.548 

152.489 

802 

3  06 


Table  III:  Effects  on  depth  of  information  processing  (continued) 


Dependent 

Independent      |  Totals: 

Meta 

n=ll 
mean=24.317 

r=ll 
mean=27.28 

n=22 
rnean=25.797 

Non-Meta 

n=ll 
mean=22,366 

n=ll 
mean=23,5 

n=22 
mean=22.933 

Totals: 

n=22 
mean=23.34 

n=22 
mean=25.39 

n=44 
mean=24.365 

*time  (seconds)  deveoted  to  reading  relevant  and  key  information 


Table  IVa:  Correlation  between  number  of  cards  (X)  an  i  recall  of  content 


Count 

Covariance 

Correlation 

R-Squared 

44 

76.07 

0.265 

0.07 

Table  IVb:  Correlation  between  number  of  cards  <X)  and  cognitive 

complexity(Y) 

Count 

Covariance 

Correlation 

R-Squared 

44 

27.121 

0.294 

0.087 

Table  V:  Effects  on  time  on  searching  for  a  topic 


Source 

DF 

Sum  of 
Squares 

Mean  Square 

F-test 

P  value 

Treatment 

1 

66.568 

66.568 

1.368 

0.249 

cognitive 
Styles  (B) 

1 

81.045 

81.045 

1.666 

0.204 

1 

32.618 

32.618 

0.67 

0.4177 

1  Error 

40 

1945.913 

48.648 

The  AB  Incidence  table  on  Y:  time  on  searching  for  a  topic 


Dependent 

Independent      1  Totals: 

Meta 

n=ll 
mean=ll,282 

n=ll 
mean=6.845 

n=22 
mean=9.064 

Non-Meta 

n=ll 
mean=7.1 

n=ll 
mean=6.107 

n=22 
mean=6.604 

Totals: 

n=22 
mean=9.191 

n=22 
mean=6.476 

n=44 
mean=7.384 

*time  (seconds)  < 

eveoted  to  searching  for  a  topic  on  main  menu  level 

Table  VI:  Effects  on  the  monitoring  guidance 
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ANOVA  table  for  a  2-factor  Analysis  of  Variance  on  Y 


Source 

DF 

Sum  of 
Squares 

Mean  Square 

F-test 

P  value 

Treatment 
(A) 

1 

29.455 

29.455 

1.659 

0.205 

cognitive 
Styles  (B) 

1 

56.818 

56.818 

3.199 

0.081 

AB 

1 

2.273 

2.273 

0.128 

0.7224 

Error 

40 

710.364 

17.759 

Table  VI:  Effects  on  the  monitoring  guidance  (continued) 


The  AB  Incidence  table  on  Y:  monitoring  guidance 


 1 

Dependent 

Independent       |  Totals: 

Meta 

n=ll 
mean=7.717 

n=ll 
mean=5 

n=22 
mean=6.364 

Non-Meta 

n=ll 
mean=5.636 

n=ll 
mean=3.818 

n=22 
mean=4.727 

Totals: 

n=22 
mean=6.682 

n=22 
mean=4.409 

n=44 
mean=5.545 

*frequent  check  of  guided  questions  and  mission 
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I  listory  can  he  viewed  as  an  interaction.  It  can  be  the  interaction  between  the  empirical  ^-id 
the  hcrmenuetical  (the  goal-to  understand),  and  the  interaction  of  science  and  art,  analysis 
and  expreHHion*  description  and  interpretation,  and  events  and  thoughts, 

Thin  H\\u\y  in  an  intentional  explanation  James  D.  Finn's  thinking  which  had  an  enormous 
influence  un  the  development  of  the  process  view  of  educational  technology.  An  intentional 
explanation  focuses  on  the  thoughts  and  conscious  activities  of  an  individual,  or  group  of 
imiividtiaiR,  and  examines  the  influence  that  these  thoughts  and  activities  had  on  later 
historical  events  and  outcomes. 

Finn'n  contribution  to  the  development  of  the  process  view  of  educational  technology  can  best 
he  analyzed  in  light  of  his  desire  to  upgrade  the  status  of  the  audiovisual  education 
movement  to  a  professional  field  of  study.  Finn's  influence  on  the  first  definition  of  the  field 
iti  1963  and  his  effort  to  change  the  name  of  the  field  to  educational  technology  in  1965  are 
jtmt  Home  of  the  examples  of  his  contribution  to  the  process  view  of  educational  technology. 

Finn  covered  much  ground  in  his  writing  about  the  techniques  of  the  field  •  the  systems 
eotwept,  the  importance  of  programming,  the  difference  between  a  scientific  and  an  empirical 
approach  to  developing  instruction  -.to  name  but  a  few.  But  Finn's  interest  in  the  process 
view  of  the  field  can  be  traced  to  his  earliest  writings.  In  1953  Finn  evaluated  the 
audiovisual  field  as  a  candidate  for  professional  status.  He  identified  six  characteristics  of  a 
profesHion: 

(a)  an  intellectual  technique,  (b)  an  application  of  that  technique  to  the 
pructical  affairs  of  man,  (c)  a  period  of  long  training  necessary  before  entering 
into  the  profession,  (d)  an  association  of  the  members  of  the  profession  into  a 
closely-knit  group  with  a  high  quality  of  communication  between  members, 
(e)  a  series  of  standards  and  a  statement  of  ethics  which  is  enforced,  and  (f) 
an  organized  body  of  intellectual  theory  constantly  expanded  by  research 
(Finn,  1953  p.7). 

Of  these  six  characteristics  of  a  profession  the  two  that  were  most  important  to  the 
development  of  Finn's  process  view  of  educational  technology  were  (1)  an  intellectual 
technique  and  (2)  a  body  of  intellectual  theory  expanded  by  research.  In  fact,  Finn  sought  to 
have  the  intellectual  technique  of  the  field  based  on  research  and  theory. 

Finn  (1953)  argued  that  "the  most  fundamental  and  most  important  characteristic  of  a 
profession  is  that  the  skills  involved  are  founded  upon  a  body  of  intellectual  theory  and 
research"  (p.8).  Once  he  had  established  the  importance  of  systematic  theory  and  research 
for  a  profession,  Finn  further  explained  his  position  by  saying  that  "...this  systematic  theory 
is  constantly  being  expanded  by  research  and  thinking  within  the  profession"  (p.8).  Finn  was 
arguing  that  a  profession  conducts  its  own  research  and  theory  development  to  complement 
the  research  and  theory  that  it  adapts  or  adopts  from  other  academic  areas  of  study. 

In  1953  Finn  was  generally  satisfied  that  "the  audiovisual  member  does  possess  an 
intellectual  technique"  (p.8),  but  it  was  not  clear  just  what  that  technique  was.  Finn 
described  the  technique  as  thinking  "reflectively  in  such  varied  areas  as  the  critical 
evaluation  of  materials,  the  visualization  of  abstract  concepts,  the  improvement  of 
instruction,  and  in  many  aspects  of  planning  and  administration"  (p.8).  Finn  did  not  further 
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describe  what  he  meant  by  thinking  reflect  ively,  It  wan  not  clear  whether  he  was  using  the 
phrase  as  John  Dewey  had,  in  relation  to  tin*  scientific  method  (Finn  was  certainly  familiar 
with  the  work  of  John  Dewey),  or  if  ho  mount  it  as  something  altogether  different.  It  seems 
likely  that  he  meant  it  as  the  application  of  research  mid  theory  to  professional  practice. 

But  Finn  believed  that  the  audiovisual  field  wan  plagued  by  a  "lack  of  theoretical  direction" 
(1953,  p. 14).  lie  attributed  thin  to  a  general  "hick  of  content"  (p.  14)  in  the  field.  He  also 
charged  that  there  was  un  absence  of  "intellectual  meat"  (p. 14)  in  the  meetings  and 
professional  journals  of  the  field.  In  his  argument  promoting  the  development  of  a 
theoretical  baso  for  the  audiovisual  field,  Finn  warned  that: 

Without  a  theory  which  produces  hypotheses  for  research,  there  can  be  no 
expanding  knowledge  and  technique.  And  without  a  constant  attempt  to 
assess  practice  so  that  the  theoretical  implications  may  be  teased  out,  there 
can  be  no  assurance  that  we  will  ever  have  a  theory  or  that  our  practice  will 
make  sense  (1953,  p.  14). 

Finn  spent  much  of  his  career  rectifying  this  deficiency  in  research  and  theory  in  the 
audiovisual  field. 

Whether  one  agrees  with  Finn's  criteria  and  reasoning  about  professions  in  general,  or  the 
audiovisual  profession  in  particular,  is  not  the  issue  here.  What  is  important  is  that  this 
argument  was  largely  accepted  by,  and  had  a  profound  effect  on,  the  leadership  of  the 
audiovisual  field.  It  also  established  a  direction  for  Finn's  future  research  and  scholarship. 
The  view  that  technology  was  primarily  a  process  was  a  favorite  theme  of  Finn.  Although  he 
was  the  acknowledged  leader  of  the  early  "educational  technology  is  a  process"  movement 
(Heinich,  1968;  Ely,  1970;  AECT,  1977),  Finn  was  not  always  consistent  about  the  label  that 
he  used  to  describe  what  he  meant  by  the  complex  processes  involved  in  audiovisual 
education. 

Finn's  earlier  writings  (1955  1957)  concentrated  on  the  concept  of  automation  in  education. 
Analyzing  the  possibilities  of  automation  for  education,  Finn  wrote  that  "...automation  is  not 
a  manless  [sic],  machine-operated  production.  Ita  primary  characteristic  is  a  process  -  o  way 
of  thinking  involving  patterns  and  self-regulation  (my  emphasis).  It  is  here  that  the 
educational  implications  are  tremendous"  (1955,  p.  145), 

Two  years  later  Finn  produced  the  first  in  a  series  of  three  articles  about  the  potential  for 
automation  in  education  for  A  V  Communication  Review,  "Automation  and  Education: 
General  Aspects"  (1957).  In  that  article  Finn  identified  the  important  characteristics  of 
automation  and  its  associated  processes  as:  "..(a)  the  concept  of  systems;  (b)  the  flow  and 
control  of  information;  (c)  scientific  analysis  and  long-range  planning;  (d)  an  increase  in  the 
need  for  wise  decision  making;  and,  (c)  a  high-level  technology"  (pp.1 15-116). 

Initially,  Finn  considered  automation  to  be  an  expansion,  an  outgrowth,  of  technology.  The 
fifth  characteristic  of  automation  that  Finn  identified,  that  automation  included  "a  high-level 
technology",  showed  that  at  this  point  in  his  thinking  Finn  drew  a  distinction  between 
automation  and  technology.  Soon,  however,  this  distinction  became  blurred. 

Finn  frequently  used  the  terms  "technical"  and  "technology"  in  his  early  writings  (1953,  1955, 
1956).  But  it  was  in  the  early  1960's  when  Finn  changed  the  focus  of  those  educational 
processes  which  interested  him  from  automation  to  technology. 
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Finn  spent  a  substantial  part  of  his  professional  life  trying  to  dispel  the  image  that 
technology  was  just  machines.  In  1960  (b)  Finn  wrote  that  "technology,  however,  is  more 
than  an  invention  -  more  than  machines.  It  is  a  process  and  a  way  of  thinking"  (p.  142). 
Continuing,  Finn  explained  the  relationship  of  technology  to  the  instructional  process  by 
saying 

(one)  must  remember  that,  in  addition  to  machinery,  technology  includes 
processes,  systems,  management,  and  control  mechanisms  both  human  and 
non  human,  and  above  all,  the  attitude  discussed  by  (Charles)  Beard  -  a  way 
of  looking  at  problems  as  to  their  interest  and  difficulty  (broadly  conceived)  of 
those  solutions.  This  is  the  context  in  which  the  educator  must  study 
technology  (p.  145). 

In  1961  Finn  made  only  minor  revisions  of  his  description  of  technology.  In  an  article 
analyzing  the  audiovisual  needs  for  the  preparation  of  teachers,  he  argued  that 
"technology... is  much  more  than  machines;  technology  involves  systems,  control  mechanisms, 
patterns  of  organization,  and  a  way  of  approaching  problems"  (Finn,  1961  p.209). 

In  opposition  to  the  view  that  technology  is  a  device  or  series  of  devices,  Finn  argued  that 
machines  were  "symbols. ..and  must  be  thought  of  in  connection  with  systems,  organizational 
patterns,  utilization  practices,  and  so  forth,  to  present  a  true  technological  picture"  (Finn, 
1961  p.210).  And  later,  as  part  of  a  speech  delivered  to  the  John  Dewey  Society  in  1962,  Finn 
said  that  "technology  is  not,  as  many  of  the  technically  illiterate  seem  to  think,  a  collection  of 
gadgets,  of  hardware,  of  instrumentation.  It  is  instead,  best  described  as  a  way  of  thinking 
about  certain  classes  of  problems  and  their  solutions"  (Finn,  1962  p.70). 

In  1964  he  wrote  that  "Technology  is  not  just  hardware-or  even  hardware  and  materials. 
Technology  is  a  way  of  organizing,  a  way  of  thinking,  involving  at  the  center,  to  be  sure,  man- 
machine  systems,  but  including  systems  of  organization,  patterns  of  use,  tests  of  economic 
feasability  "(1964,  p.295). 

In  1966  (b)  he  followed  that  with  "Instructional  technology  [should]  be  viewed  as 
defined  in  this  paper-a  complex  pattern  of  man-machine  systems  and  organizations 
based  on  concepts  of  feasibility"  (Finn,  1966b,  p. 247). 

It  is  unclear  if  Finn  decided  that  technology,  which  in  1957  he  considered  a  condition  of 
automation,  had  to  be  determined  and  clarified  as  part  of  a  desired  automated  state  of 
education,  or  if  Finn  decided  to  change  the  label  of  the  object  that  he  was  talking  about  all 
along,  "automation"  to  "technology".  In  either  case,  there  are  certain  themes  which 
continued  to  appear  throughout  his  writing. 

There  is  a  great  deal  of  similarity  and  consistency  about  the  way  in  which  Finn  described 
automation  and  technology.  He  argued  that  automation  was  a  process  and  a  way  of  thinking 
(1955,  1957)  and  that  technology  was  a  process  and  a  way  of  thinking  (1960b,  1961).  He 
further  stated  that  automation  included  systems,  controls,  scientific  analysis  and  planning 
(1955,  1957)  and  that  technology  included  systems,  controls,  and  management  and/or 
organization  (1960a,  1961).  Finn's  discussions  of  scientific  analysis  and  planning  in 
automation  (1957)  are  very  similar  to  his  discussions  of  management  and  organization 
included  in  technology  (1961,  1962). 


While  Finn's  descriptions  of  automation  and  technology  remained  similar  over  the  years, 
there  is  one  important  facet  that  did  seem  to  shift  within  Finn's  writing  on  technology:  the 
relationship  of  technology  to  problems.  In  1960  (a)  Finn  viewed  technology  as  "a  way  of 
looking  at  problems1'.  In  1961  Finn  stated  that  technology  is  "a  way  of  approaching 
problems".  It  could  be  argued  that  "looking  at  problems"  means  defining  problems,  and  that 
"approaching  problems"  means  defining  problems  but  also  taking  some  action  to  solve  a 
problem  as  well.  The  phrases  "looking  at  problems"  and  "approaching  problems"  both  infer  a 
way  to  begin  to  solve  problems.  Both  phrases  infer  a  certain  "attitude"  (1960a)  toward 
professional  practice.  This  attitude  would  direct  the  intellectual  technique  of  the  profession. 

In  1962  there  is  a  not-so-subtle  shift  that  occurs  in  Finn's  outlook  which  carried  on  through 
his  subsequent  discussions  (1965b,  1966b)  of  technology  and  problem  solving.  It  was  the 
view  that  technology  is  "a  way  of  thinking  about  certain  classes  (my  emphasis)  of  problems". 
It  is  clear  in  this  statement  that  there  were  limitations  to  technology's  problem  solving 
ability.  It  seems  to  mean  that  technology  either  could  not  solve,  or  should  not  be  used  to 
solve,  all  problems.  Considering  Finn's  optimistic  view  of  technology  one  could  easily 
conclude  that  his  position  prior  to  1962  was  that  technology  was  essentially  the  technique 
which  ould  be  used  to  look  at  or  approach  all  educational  problems. 

It  is  difficult  to  assess  what  prompted  Finn  to  make  this  further  clarification  in  his  position. 
Perhaps  it  was  the  fact  that  this  1962  statement  was  part  of  his  address  to  the  John  Dewey 
Society.  The  prospect  of  talking  with  educational  philosophers  may  have  provided  Finn  with 
reason  to  reflect  on  his  ideas  concerning  technology  as  a  way  of  thinking.  What  is  certain  is 
that  this  shift  in  Finn's  Writing  on  technology  remained  for  the  rest  of  his  professional  life. 

The  implication  of  this  shift  by  Finn  is  not  recognized  by  those  members  of  the  field  who 
describe  educational  technology  as  simply  "a  problem  solving  approach  to  education".  It  does 
not  seem  as  if  Finn  intended  educational  technology  to  be  viewed  as  an  all  encompassing 
approach  to  education  as  was  inferred  by  later  definitions  of  our  field. 

Another  look  at  his  analyses  of  technology  reveals  that  even  Finn,  the  great  proponent  of 
technology  as  process,  conceded  that  machines  and  materials  were  a  major  part  of 
instructional  technology.  This  is  best  seen  in  his  discussion  (1965a)  which  was  aimed  at 
changing  the  name  of  the  professional  organization  and  the  field  to  instructional  technology. 
There  he  stated  that  "machines,  materials,  methods  of  use,  [and]  systems  are  all  part  of  the 
pattern  of  rational  mechanisms  operating  as  means  to  educational  ends,  And,  as  Hoban  has 
said,  machines  are  central  to  this  concept  even  though  they  alone  are  not  technology"  (Finn, 
1965a,  p. 193).  This  statement  is  consistant  with  his  prior  writings  on  technology,  nil  of 
which  include  machines  as  an  essential  component  of  technology.  For  Finn,  it  seemed  that  if 
there  were  no  machines  there  could  be  no  technology. 

Conclusion 

There  are  three  conclusions  which  can  be  drawn  from  this  brief  exploration  of  Finn's  writing 
about  the  intellectual  technique  of  our  field:  1)  Finn  was  a  man  with  a  vision  for  his 
profession.  But  as  is  often  the  case  those  with  long  term  visions  they  focus  on  broad  ideas 
and  do  not  always  develop  those  ideas  into  specific  concepts.  This  seems  to  be  the  case  with 
Finn  as  we  never  get  a  specific  statement  of  his  definition  of  technology;  2)  Finn  changed  his 
ideas  over  time  ,  but  so  do  we  all.  Here  it  is  exemplified  by  his  discusions  on  problems  and 
the  terms  automation  and  technology;  and  3)  Finn  may  have  been  misinterpreted  by  others 
in  our  field.  What  is  certain  is  that  ideas  which  he  himself  articulated  were  changed  or 
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reoriented  by  the  membership  of  the  field.  This  is  simply  a  playing  out  of  the  premise  that 
ideas  change  over  time. 

No  matter  how  one  views  his  writing  and  his  influence  on  the  development  of  educational 
technology  Finn  had  the  long  range  vision  which  gave  direction  to  his  work,  and  he  had  the 
drive  to  gain  that  direction-  to  professionalize  the  audiovisual  field. 
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the  bonk  KxtontlitiH  Kduvutum  Through  7ta7t/io/ofty,  nvlvvtvd  writin#H  byJamvs  D. 
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have  cheeked  many  of  the  pupct'H  of 4 his  volume  agahiHt  the  original  and  finds  them 
to  be  must  accurate  Kur  practical  popones,  In  Huh  study  I  have  used  that  volume 
oiHfeod  of  many  of  the  original  artlciew 

Mini's  six  ehoraetoHwIlcH  uf  a  piofosMhini 


(a)  an  Intellectual  technique, 

(b)  an  application  of  that  technique  to  the  practical  affairs  of  man, 

(c)  a  period  of  long  training  necessary  before  entering  into  the 
profession, 

(d)  an  association  of  the  members  of  the  profession  into  a  closely- 
knit  group  with  a  high  quality  of  communication  between 
members, 

(e)  a  series  of  standards  and  a  statement  of  ethics  which  is 
enforced, 


(0     an  organized  body  of  intellectual  theory  constantly  expanded 
by  research 
(Finn,  1953  p.7). 


The  two  that  were  most  important  to  the  development  of  Finn's  process  view  of  educational 
technology  were  (1)  an  intellectual  technique  and  (2)  a  body  of  intellectual  theory  expanded 


by  research.  In  fact,  Finn  sought  to  have  the  intellectual  technique  of  the  field  based  on 
research  and  theory. 

Quotes 

"...automation  is  not  a  manless  [sic],  machine-operated  production.  Its  primary  chcracterintic 
is  a  process  -  a  way  of  thinking  involving  patterns  and  self-regulation  (my  emphasis).  It  is 
here  that  the  educational  implications  are  tremendous"  (1955,  p.  145). 

"..(a)  the  concept  of  systems;  (b)  the  flow  and  control  of  information;  (c)  5ci9ntific  analysis  and 
long-range  planning;  (d)  an  increase  in  the  need  for  wise  decision  making;  and,  (e)  a  high- 
level  technology"  (1957  pp.  115- 116). 

"technology,  however,  is  more  than  an  invention  -  more  than  machines.  It  is  a  process  and  & 

way  of  thinking"  (1960,  p.  142).  Continuing,  Finn  explained 

(one)  must  remember  that,  in  addition  to  machinery,  technology  includes 
processes,  systems,  management,  and  control  mechanisms  both  human  and  non 
human,  and  abo>?  all,  the  attitude  discussed  by  (Charles)  Beard  •  a  way  of 
looking  at  problems  as  to  their  interest  and  difficulty  (broadly  conceived)  of  those 
solutions.  This  is  the  context  in  wnich  the  educator  must  study  technology  (I960, 
p.  145). 

"technology... is  much  more  than  machines;  technology  involves  systems,  control  mechanisms, 
patterns  of  organization,  and  a  way  of  approaching  problems"  (Finn,  19(U  p.2()9).. .machines 
were  "symbols. ..and  must  be  thought  of  in  connection  with  systems,  ortfanizot  ;onnl  patterns, 
utilization  practices,  and  so  forth,  to  present  a  true  technological  picture"  (Finn,  1961  p.210). 
"technology  is  not,  as  many  of  the  technically  illiterate  seem  to  think,  n  collection  of  gadgets, 
of  hardware,  of  instrumentation.  It  is  instead,  best  described  as  a  way  of  thinking  about 
certain  classes  of  problems  and  their  solutions"  (Finn,  1962  p. 70). 

"Technology  is  not  just  hardwarc-or  even  hardware  and  materials.  Technology  is  a  way  of 
organizing,  a  wav  of  thinking,  involving  at  the  center,  to  be  sure,  man-machine  systems,  but 
including  systems  of  organization,  pattern*  of  use,  tests  of  economic  feasabilit>"  (1964  P.295) 
Instructional  technology  (should]  bo  viewed     defined  in  this  paper- a  complex  pattern  of 
man-machine  systems  and  organizations  ba"ed  on  concepts  of  feasibility  and  th*t  proposals 
centering  on  hardware  and  materials  be  required  to  show  some  understanding  ot  this  concept 
on  an  operational  basis.  (Finn,  1966  p.247} 

"This  has  resulted  in  a  technological  c?  empirical  approach  to  solving  practical  problems  of 
teaching  and  learning  "  (1966,  p.283) 

Conclusions 

1)  Finn  was  a  man  with  a  vision  foi  his  profession.  But  as  is  often  the  case  those 
with  long  term  visions  they  focus  on  broad  ideas  and  do  not  a* ways  develop 
those  ideas  into  specific  concepts.  This  seems  to  be  the  C03e  with  Finn  as  we 
never  get  a  specific  statement  of  his  definition  of  technology; 

2)  Finn  changed  his  ideas  over  time.  This  is  exemplified  by  his  discusions  on 
problems  and  the  terms  automation  and  technology,  but  so  do  we  all; 

3)  Finn  may  have  been  misinterpreted  by  others  in  our  Held.  What  is  certain  is 
that  ideas  which  he  himself  articulated  were  changed  or  reoriented  by  the 
membership  of  the  field.  This  is  simply  a  playing  r  ut  of  the  premise  that  ideas 
change  over  time. 

No  matter  how  one  views  his  writing  and  hL  influence  on  the  development  of 
educational  technology,  Finn  had  tha  long  range  vision  which  gave  direction  to  his 
work  and  he  had  the  drive  to  gain  chat  direction-  to  professionalize  the 
audiovisual  field. 
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In  recent  yearn,  I'iilin  itt tmiiil  Tei -IiihiIumIhIh  have  oxaanned  hnw  to  iinplrinrnt 
cooperative  learning  h! rnli*|fii-H  in  net!  I mmm  1  tin t  wen*  niiunially  denigned  for  individual 
learning.  Classroom  rewari  |i  gonerullv  mdicaten  |nmltive  rcnallM  in  favor  of  cooperative 
learning  (Johnson  &  iIoIiiihiiii,  I  HMD,  HIiiii tin,  IHHM;  HIiivIii,  IIIIMH.  However,  research  that 
examines  how  to  implement  c'iiii|ii*rrit lvi»  learning  with  media  hnn  produced  mixed  results, 
Some  researches  report  that  cooperative  learning  ponltivoly  affected  performance  in  CAl 
lessons  (Dalton,  Hannafin,  Hi  Hooper,  I  OH! I;  JiiIiiihiiii,  Johanna,  &  Htannc,  1985),  while 
others  have  not  found  a  significant  vftvvl  lor  performance  when  learners  lined  cooperative 
CAI  (Carrier  &  Sales,  1987),  Research  on  cooperative  learning  and  inHtnii'tioiial  television 
suggests  that  when  compared  to  individual  strategies,  cooperative  learning  does  not  always 
increase  learning  and  motivation  (Klein,  Kivhu),  &  I'rideuinro,  MMKl).  Other  Htudies  report 
that  performance  in  these  setting  iH  intlueiieed  liy  iimo'h  affiliation  motives  (Klein  & 
Pridemore,  1992). 

Results  obtained  from  using  cooperative  learning  with  media  may  be  influenced  by 
the  instructional  strategies  employed  in  these  HtudieH.  Different  instructional  elements 
incorporated  within  lessons  could  influence  the  effectiveness  of  using  cooperative  learning 
with  these  media.  Two  instructional  elements  which  may  influence  the  success  of 
cooperative  learning  with  media  are  orienting  activities  and  practice.  Both  have  influenced 
learning  outcomes  in  a  variety  of  settings  (e.g.,  Ausubel,  1960;  Ho,  Savenye,  &  Haas,  1986; 
Phillips,  Hannafin,  &  Tripp,  1988;  Salisbury,  Richards,  &  Klein,  1985). 

The  purpose  of  the  current  study  was  to  investigate  the  effects  of  orienting  activities 
and  practice  on  performance  and  student  behaviors  in  a  cooperative  learning  environment. 

Method 

Design.  A  2  X  2  factorial  design  was  used  in  this  study,  with  orienting  activity 
(advance  organizer  versus  objectives)  and  type  of  practice  (verbal  information  versus 
intellectual  skill)  as  the  independent  variables.  The  dependent  variables  were  achievement 
and  student  behaviors. 

Subjects.  Subjects  were  80  graduate  education  majors  (16  males,  64  females)  at  a 
large  southwestern  university  enrolled  in  a  required  course  in  educational  psychology. 
Subjects  were  stratified  by  sex  and  randomly  assigned  to  one  of  16  cooperative  learning 
groups.  Bach  group  was  constructed  to  include  one  male  and  four  female  students.  Each 
group  was  randomly  assigned  to  one  of  four  instructional  treatment6  (advance 
organizers/information  practice,  advance  organizers/skills  practice,  objectives/  information 
practice,  objectives/skills  practice). 

Materials  &  Procedures.  All  groups  received  three  instructional  television  lessons 
from  the  series  Instructional  Theory:  A  nine  lesson  mini-murse  (Gerlach,  1973).  Each  lesson 
included  a  videotape  and  a  workbook  that  provided  instruction  on  the  topics  of  stating 
objectives,  developing  assessment  items,  and  designing  effective  practice  activities.  Each 
videotape  was  divided  into  several  segments  which  presented  information  and  examples  on 
the  content  of  the  lesson.  After  each  segment,  the  videotape  instructed  subjects  to  turn  to  a 
workbook  for  practice  and  feedback  on  the  content  presented  in  that  segment.  Variations  in 
the  materials  occurred  in  the  workbooks.  Each  workbook  provided  either  an  advance 
organizer  or  a  list  of  instructional  objectives  for  the  lesson.  The  advance  organizer  provided 
students  with  an  overview  of  the  lesson  for  that  day.  The  following  is  an  example  of  the 
advance  organizer  provided  to  students  for  the  lesson  on  writing  objectives. 
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Perhaps  the  best  known  component  of  the  systematic  approach 
to  instructional  design  is  the  instructional  objective. 
Since  the  early  1 960*8,  many  educators  have  written 
instructional  objectives.  An  objective  is  a  statement  of 
what  students  will  be  able  to  do  after  they  have  completed 
a  unit  of  instruction.  In  this  lesson  you  will  be  learning 
how  to  use  objectives  in  order  to  improve  your  instruction. 

The  other  set  of  workbooks  provided  a  specific  list  of  the  instructional  objectives  for  each 
lesson.  The  following  is  an  example  of  the  objectives  provided  to  students  for  the  lesson  on 
writing  objectives. 

1 .  Explain  why  well  stated  objectives  are  essential  to  the 
devolopment  of  effective  instruction. 

2.  Identify  examples  of  properly  stated  objectives,  given 
examples  of  properly  and  improperly  stated  objectives. 

3.  Name  the  desired  characteristics  of  objectives  that  are 
missing,  given  an  improperly  stated  objective. 

The  workbooks  for  each  lesson  also  differed  on  the  type  of  practice  given  after  each 
video  segment.  Eight  of  the  16  cooperative  learning  groups  used  workbooks  that  provided 
practice  on  verbal  information  presented  in  each  videotape.  The  other  eight  groups  used 
workbooks  that  provided  practice  on  the  intellectual  skills.  For  example,  Segment  4  of  the  ' 
lesson  on  objectives-based  assessment  provided  information  and  examples  on  paper-and- 
pencil  tests,  interviews,  and  observations  of  student  performance  or  product.  Practice  on  the 
verbal  information  for  this  segment  required  students  to  summarize  the  key  information  in 
the  videotape  by  listing  the  three  types  of  objectives-based  assessment.  Practice  on  the 
intellectual  skills  for  this  segment  required  students  to  describe  the  best  type  of  objectives- 
based  assessment  for  the  objective  of  sculpting  a  human  head  based  on  criteria  discussed  in 
an  art  class.  After  each  practice  question,  written  feedback  was  provided. 

Students  were  provided  with  each  lesson  over  three  consecutive  days.  While 
students  worked  through  the  lesson,  an  observer  watched  the  groups  and  recorded  student 
behaviors  Three  days  later,  each  student  completed  a  constructed  response  posttest  in 
their  regular  class  to  evaluate  individual  student  mastery  of  the  three  lessons. 

Criterion  Measures.  A  paper-and-pencil  posttest  that  consisted  of  22  short -response 
items  was  used  to  measure  performance.  This  posttest  was  divided  into  two  sections.  The 
first  section  contained  14  intellectual  skill  items  to  evaluate  application  of  lesson  content. 
The  second  section  of  the  posttest  contained  eight  verbal  information  items  to  evaluate 
knowledge  of  lesson  content.  Both  sections  of  this  test  were  worth  a  total  of  twenty  points 
each.  Individual  answers  to  each  item  were  checked  against  a  scoring  key  and  points  were 
assigned  for  each  response.  An  item  was  worth  one  point  unless  it  required  a  multiple 
response.  One  person  scored  all  of  the  items  on  this  test.  The  Kuder-Richardson  internal- 
consistency  reliability  of  the  posttest  was  .74  for  subjects  in  the  present  study. 

Student  behaviors  were  observed  during  each  of  the  three  lessons  and  field  notes 
were  recorded  by  two  observers.  After  data  collection  had  occurred,  a  classification  scheme 
tor  the  field  notes  was  developed  based  on  research  findings  from  Webb  0982,  1987).  Webb 
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suggested  that  student  behaviors  in  small  groups  can  involve  on-task  versus  off-task 
interactions,  helping  behaviors,  and  working  alone.  The  classification  scheme  developed  for 
the  current  study  identified  four  sets  of  student  behaviors,  including  (1)  helping  behavior 
(asking  for  help,  giving  help  when  asked,  giving  unsolicited  help),  (2)  on-task  group  behavior 
(taking  turns,  sharing  materials,  group  discussion  of  content),  (3)  on-task  individual 
behavior  (assuming  control,  taking  notes,  working  alone),  and  (4)  off-task  behavior  (talking 
to  other  about  something  unrelated  to  the  lessons  and  non-verbal  actions  such  as  reading  a 
newspaper). 

Using  this  classification  scheme,  both  observers  examined  all  of  the  field  notes  to 
calculate  the  total  number  of  times  these  behaviors  occurred  for  each  of  the  16  cooperative 
learning  groups.  Reliability  of  observations  was  based  on  both  observers  having  similar 
totals  for  each  set  of  student  behaviors.  The  inter-rater  reliability  between  observers  was 
.84  for  helping  behaviors,  .84  for  on-task  group  behaviors,  .79  for  on-task  individual 
behaviors,  and  .82  for  off-task  behaviors. 

Data  Analysis.  MANOVA  was  used  to  analyze  performance  data.  Individual 
student  scores  on  both  sections  of  the  posttest  were  combined  for  this  analysis.  MANOVA 
was  followed  by  univariate  analyses  on  the  knowledge  and  application  portions  of  the 
posttest.  Alpha  was  set  at  .05  for  these  statistical  tests.  MANOVA  was  also  used  to 
analyze  each  of  the  four  sets  of  student  behaviors.  These  behaviors  were  considered  as  a 
group-based  measure  in  these  analyses  since  a  combined  score  was  obtained  for  subjects  in 
each  cooperative  learning  group.  Alpha  was  set  at  .05  for  the  multivariate  tests.  MANOVA 
was  followed  by  univariate  analyses  on  the  student  behaviors  in  a  set.  To  account  for  the 
possibility  of  inflated  statistical  error,  alpha  was  set  at  .015  for  the  univariate  analyses. 

Results 

Performance.  MANOVA  revealed  a  significant  interaction  between  orienting  activity 
and  type  of  practice,  [E(2,  70)  =  3.87,  £  <  .05].  Univariate  analyses  revealed  that  the 
interaction  was  significant  for  the  knowledge  portion  of  the  posttest,  [£(1,  71)  =  4.70,  j& 
<.05J.  This  interaction  was  disordinal  in  nature.  Sheffe*  multiple  comparison  tests  revealed 
that  subjects  who  received  objectives  and  information  practice  obtained  significantly  more 
knowledge  (M  =  11.6)  than  those  who  received  advance  organizers  and  verbal  information 
practice  (M  =  9.4),  [£(3,  75)  =  3.27,  fi  <  .05].  Sheffe*  tests  also  indicated  that  subjects  who 
received  objectives  and  information  practice  obtained  significantly  more  knowledge  (M  = 
11.6)  than  those  who  received  objectives  and  intellectual  skills  practice  (M  =  9.3),  [£(3,  75)  = 
3.57,  u  <  .05]. 

In  addition  to  the  interaction,  MANOVA  indicated  a  significant  main  effect  for  type  of 
practice,  [E(2,  70)  =  6.84,  b  <  05],  but  not  for  orienting  activity.  Subjects  who  worked  in 
groups  that  received  skills  practice  achiever'  higher  overall  posttest  scores  (M  =  23.4)  than 
those  who  received  information  practice  (M  =  21.4).  Univariate  analyses  revealed  that 
subjects  who  received  practice  on  the  intellectual  skills  presented  in  the  lessons  performed 
significantly  better  on  the  application  portion  of  the  posttest  (M  =  13.0)  than  subjects  who 
received  practice  on  the  verbal  information  (M  =  10.9),  [E(l,  71)  =  8.54,  u  <  .05]. 

Helping  Behaviors.  MANOVA  indicated  that  the  type  of  practice  given  to  students 
significantly  affected  their  helping  behaviors,  [E(3,  10)  =  S.  39,  y  <  .05].  Follow-up  univariate 
analyses  indicated  that  students  who  received  practice  over  the  skills  presented  in  the 
lesson  gave  significantly  more  help  to  their  fellow  group  members  (M  =  2.13)  than  students 
who  received  practice  over  the  information  (M  =  0.5),  [E(l,  12)  =  15,36,  jj  <  .01]. 
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On-task  Group  Behaviors.  MANOVA  indicated  that  type  of  practice  [£(3,  10)  =  7.28, 
£  <  .01]  and  orienting  activity  [£(3,  10)  =  4.23,  ft  <  .Ofj]  had  a  significant  effect  on  students1 
on-task  group  behaviors.  Follow-up  univariate  analyses  indicated  that  students  who 
received  skills  practice  discussed  significantly  more  of  the  lesson  (M  =  9.75)  than  students 
who  received  information  practice  (M  =  4.88),  [£(1,  12)  =  21.83,  ft  <  .01].  In  addition, 
students  who  received  objectives  discussed  significantly  more  of  the  lesson  (M  =  9-25)  than 
students  who  received  advance  organizers  (M  =  5.34),  [£(1,  12)  =  13.79,  ft  <  .01]. 

Qn-task  Individual  Behaviors.  MANOVA  indicated  that  the  type  of  practice  given  to 
students  significantly  affected  their  on-task  individual  behaviors,  [E(3,  10)  =  18.26,  ft  <  .05], 
Follow-up  univariate  analyses  indicated  that  students  who  received  information  practice 
took  more  notes  on  the  lesson  (M  =  6.63)  than  students  who  received  skills  practice  (M  = 
0.5),  [£(1,  12)  =  62.63,  ft  <  .01]. 

Off-task  Behaviors.  MANOVA  did  not  indicate  a  significant  effect  for  type  of  practice 
nor  orienting  activity  when  off-task  behaviors  were  analyzed.  Furthermore,  a  significant 
interaction  between  type  of  practice  and  orienting  activity  was  not  found. 

Discussion 

The  purpose  of  this  study  was  to  investigate  the  effects  of  orienting  activities  and 
type  of  practice  on  performance  and  student  behaviors.  Subjects  assigned  to  cooperative 
learning  groups  received  information,  examples,  practice,  and  feedback  from  three 
instructional  television  lessons.  Each  lesson  included  specific  orienting  activities  (advance 
organizers  or  objectives)  and  different  types  of  practice  (verbal  information  or  intellectual 
skills). 

Results  indicated  that  the  type  f  practice  provided  to  students  had  a  significant 
effect  on  performance.  Subjects  who  worked  in  groups  that  received  skills  practice  performed 
better  than  those  who  received  information  practice.  Examination  of  each  section  of  the 
posttest  suggested  that  performance  on  application  items  was  offected  by  type  of  practice. 
While  type  of  practice  by  itself  did  not  influence  knowledge  acquisition  in  the  present  study, 
it  was  affected  by  a  combination  of  type  of  practice  and  orienting  activity.  Subjects  who 
worked  in  groups  that  received  information  practice  and  objectives  outperformed  all  other 
subjects  on  the  knowledge  portion  of  the  posttest. 

A  possible  explanation  for  these  findings  may  be  due  to  the  nature  of  the 
instructional  elements  provided  during  the  lessons  and  the  subsequent  performance 
measure.  Students  in  groups  who  received  skills  practice  had  the  opportunity  to  perform 
tasks  similar  to  those  required  on  the  application  items.  This  practice  strengthened  student 
ability  to  apply  concepts  presented  in  the  •  jssons.  However,  students  who  received 
information  practice  only  had  the  opportunity  to  rehearse  the  knowledge  provided  in  the 
lessons.  Providing  lesson  objectives  to  these  students  may  have  been  necessary  to  direct 
their  attention  and  selective  perception  to  relevant  lesson  content. 

In  addition  to  performance,  orienting  activity  and  type  of  practice  influenced  student 
behaviors  in  the  current  study.  Groups  given  objectives  discussed  significantly  more  content 
than  groups  given  advance  organizers.  This  result  may  have  occurred  because  subjects  had 
extensive  experience  with  using  instructional  objectives  as  a  study  tool  and  little  experience 
with  implementing  advance  organizers.  While  objectives  influenced  group  behaviors,  type  of 
practice  had  a  stronger  affect  on  student  behaviors.  Groups  who  received  skills  practice 
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exhibited  significantly  more  helping  behaviors,  more  on-task  group  behavior,  and  less 
individual  behavior  (taking  notes)  than  groups  who  received  knowledge  practice.  These 
results  likely  occurred  because  groups  who  were  given  skills  practice  found  the  lessons  more 
interesting  and  engaging  than  those  who  received  information  practice. 

This  study  has  some  implications  for  those  who  design  instruction.  The  current 
findings  suggest  that  instructional  technologists  can  increase  performance  and  student 
interaction  by  employing  high-level  practice  and  specific  orienting  activities  when  using 
cooperative  learning  with  media  originally  designed  for  individual  learning.  Systematic 
application  of  the  appropriate  instructional  elements  in  these  settings  will  increase  the 
success  of  cooperative  learning. 
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THE  NEED 


Professionals  who  live  in  rural  areus  often  have  difficulty  finding  opportunities  to 
receive  further  professional  development  without  spending  substantial  amounts  of  time 
traveling  to  training  sites.  In  an  attempt  to  improve  services  to  rural  populations,  distance 
education  has  become  an  increasingly  popular  form  of  continued  training  to  adult  audiences. 
The  growing  emphasis  on  distance  education  recognizes  the  needs  of  rural  professionals  who 
may  not  have  the  time  or  resources  to  travel  to  more  central  training  sites. 

As  has  been  the  case  with  others,  the  increasing  workplace  demands  on  social 
workers  requires  a  feasible  solution  to  providing  in-service  training  for  those  who  reside  in 
rural  areas.  The  workplace  demands  place  social  workers  in  the  position  of  needing  to  knew 
more  information,  yet  having  increased  responsibilities  that  make  it  more  difficult  to  leave 
the  office.  While  distance  learning  opportunities  have  multiplied  greatly  in  the  past  several 
years,  the  lack  of  relevant  course  work  and  access  to  the  technology  necessary  to  participate 
in  these  opportunities  may  prohibit  the  wide  use  of  distance  education  for  rural  social 
workers. 

The  need  to  provide  carefully  designed  training  to  meet  theses  challenges  is  critical, 
especially  in  light  of  the  limited  resources  for  staff  development  within  the  social  work 
agencies.  The  assessment  process,  therefore,  becomes  vital  in  planning  staff  training  to 
enhance  these  limited  resources  by  concentrating  them  on  the  most  essential  needs. 

Collaboration  between  two  major  institutions  in  n  rural  midwestcrn  state  identified 
specific  training  needs  for  child  welfare  workers  and  a  five-year  plan  to  meet  them.  In  1992, 
the  Kansas  Department  of  Social  and  Rehabilitation  Services  (SRS)  contracted  with  the 
Kansas  State  University  Social  Work  and  Special  Education  programs  to  develop  a  series  of 
training  modules  for  child  wolfare  workers  that  would  emphasize  the  training  needs  of 
workers  in  the  rural  parts  of  the  statu.  The  first  step  of  the  training  process  assessed  the 
workers'  perceptions  of  their  needs.  The  second  step  developed  a  plan  to  meet  those  needs. 
This  article  briefly  describes  the  assessment  process  in  a  public  child  welfare  agency  that 
included  the  workers  themselves  as  a  primary  source  of  knowledge  about  their  own  needs 
and  then  addresses  the  plan  to  provide  the  training. 

Rationale 

Of  the  several  methods  available  for  assessing  training  needs,  three  seem  pertinent 
to  the  needs  of  child  welfare  workers.  Pecora,  Dodson,  Teather,  and  Whittaker  (1983) 
describe  these  three  as  the  task-based  survey,  the  knowledge-based  survey,  and  the  worker 
ability/characteristic  survey.  Each  method  relies  primarily  on  input  from  job  analysis  data 
and  selected  groups  of  staff  members  and  outside  experts  for  the  development  of  specific 
training  needs.  These  needs  are  then  presented  to  workers  in  a  written  format.  While 
efficient,  these  methods  are  often  perceived  as  placing  further  demands  on  workers'  time 
with  dubious  promise  of  results.  Therefore,  none  of  theses  methods  met  our  criteria  that  the 
assessment  must  not  add  to  the  workers'  frustration  and  must  be  a  positive,  relevant 
experience. 

Child  welfare  workers  can  experience  a  sense  of  worth  and  involvement  when 
concepts  form  adult  learning  theory  a  d  social  work  problem-solving  methods  are 
incorporated  into  the  needs  assessment  process.  Basic  tenets  of  adult  learning  theory 
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support  the  involvement  of  workers  in  every  stage  of  planning  for  training  and  suggest  that 
workers  be  asked  directly  what  it  is  they  believe  they  need  to  know  to  perform  better 
(Knowles,  1973).  Social  work  problem-solving  methods  begin  by  establishing  rapport  and 
progress  through  identifying  a  problem,  developing  a  goal  and  plan  to  reach  it,  implementing 
the  plan,  and  evaluating  the  plan  as  well  as  the  process  (Hepworth  and  Larsen,  1986). 
Inclusiveness  and  mutuality  are  fundamental  to  the  process.  The  method  assumes  that 
clients  can  evaluate  the  accuracy  of  their  needs  and  wants,  and  can  express  these  in  the  form 
of  goals  (Fox,  1987). 

One  research  technique  that  encompasses  many  of  those  features  involves  using 
focus  groups.  In  such  a  research  technique,  a  facilitator  meets  with  small  groups  to  present 
the  project  and  solicit  input  from  the  small  group  participants.  The  resulting  information  is 
used  to  develop  effective  strategies  to  meet  the  identified  needs.  The  data  gathered  from  this 
type  of  research  can  include  qualitative  pieces  that  might  be  not  emerge  or  be  overlooked 
with  other  types  of  data  gathering. 

Methods 

Focus  groups  for  this  project  provided  the  opportunity  for  SKS  child  welfare  staff 
members  to  communicate  their  training  needs  directly.  Several  units  were  involved, 
including  protective  services,  foster  ('are,  adoption,  family-based  in-home  services,  and  family 
services. 

Subjects  for  this  study  involved  approximately  75  supervisors  and  495  social  workers, 
representing  all  twelve  districts  within  the  state.  To  reduce  the  potential  of  promoting 
resistance  or  even  hostile  feelings  that  had  in  the  past  resulted  from  forced  participation,  the 
researchers  used  a  "invitational"  type  of  selection.  Posters  announced  the  meeting  times  and 
stressed  the  voluntary  nature  of  the  focus  groups. 

Two  researchers  facilitated  each  focus  group  and  concentrated  on  leading  the  process 
in  the  following  order:  (1)  Brainstorming  topics  of  perceived  training  needs,  (2)  organizing 
those  topics  into  units  of  study,  (3)  establishing  individual  priorities,  (4)  discussing  preferred 
methods  of  receiving  training,  (5)  demonstrating  multimediated  training,  and  (6)  gathering 
participants'  feedback  about  the  focus  group  session. 

Conclusions 

Results  of  the  focus  groups  indicated  training  priorities  and  training  delivery  issues. 
The  data  on  the  firming  priorities  consisted  nf  34  lists  of  training  topics  from  the 
brainstorming  sessions,  which  had  been  organized  into  two  parts:  (1)  34  lists  of  training  units 
orders  according  to  their  mean  rating,  and  (2)  two  lists  of  20  training  unites  ranked  according 
to  the  mean  rating  as  prioritized  by  area  directors  an  social  services  chiefs.  Table  1 
summarizes  the  frequency  with  which  topics  were  listed  by  the  focus  groups. 

Data  indicated  strong  preference  for  interactive  training  that  stressed  variety.  The 
videotape  and  workbook  format  used  for  the  agency's  basic  training  was  regarded  by  most 
participants  as  n  helpful  way  to  deliver  information.  Staff  members  who  had  used  this 
curriculum  in  small-group  sessions  were  very  positive  about  the  benefits  of  having  an 
opportunity  for  interaction  when  working  through  the  material,  The  participants  seldom 
used  the  supplementary  materials  providing  with  the  training,  except  in  the  case  of  outlines 
'   concise  handouts  that  they  were  able  to  share  with  clients.  They  preferred  that  materials 
developed  of  their  own  training  be  adaptable  for  use  with  clients. 


TABLE  1,  Workers*  Training  Needs  According  to  Focus  Groups, 


TRAINING  NEED  %  OF  TIMES  TOPICS  APPEARED 


Survival  Skills  for  Workers  1 5% 

Abuse  and  Neglect  1 1 

Legal  and  Court  System  10 

Families  7 
Adolescents  7 
Oppositional  or  Resistant  Clients  7 

Interaction  and  Interviewing  Skills  (> 
Networking  and  Community  Resources  6 
Mental  Health  5 

Writing  Reports  and  Documentation  5 
Cultural  Differences  5 
Child  Development  5 

Computer  Training  4 
Understanding  SRS  System  4 
Special  Populations  and  Special  Education  3 

n=  403  items 

THE  SOLUTION 

Based  upon  these  identified  needs,  the  following  units  of  study  were  identified  as 
high  priority  training  items: 

Child  Development  Family  Based  Treatment  Strategies 

Professional  Ethics  Family  Issues  -  Separation,  Attachment 

Stress  Management  Legal  Issues 

Abuse  and  Neglect  Practice  Skills  I 

Adolescents  Practice  Skills  U 

Because  the  focus  groups  identified  a  desire  for  interactive;  training  with  n  video 
component,  the  project  directors  chose  multimediated  training  that  included  videodisc, 
computer,  and  paper  based  materials.  The  computer  offers  the  interactivity,  the  videodisc 
offers  the  video  examples,  and  the  paper  component  offers  a  workbook  filled  with  practice 
exercises,  longer  pieces  of  text  documentation,  and  short,  concise  informational  sheets  that 
can  be  sent  home  with  clients. 

It  quickly  became  clear  that  the  content  area  experts  needed  to  learn  about  the 
capabilities  and  limitations  of  videodisc  technology.  That  has  been  an  on-going  process.  At 
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first,  then*  was  ho  one  on  the  project  team  who  could  direct  the  development  of  the  videodisc 
component.  Once  someone  was  hired,  the  following  abbreviated  list  of  instructional  design 
considerations  was  implemented  within  the  design  process  for  each  of  the  videodiscs. 

•  Who  are  the  users? 

-  Background  and  educational  level? 

-  V/hat  is  their  computer  skill  level? 

-  In  what  environment  will  this  training  he  used? 

•  Independent  Modules  that  can  be  used  in  any  order 

•  Module  Product  Components 

-  Videodisc 

-  Computer  Disk 

-  Workbook  Documentation  in  3-ring  binder  for  easy  removal  of  pages 

-  Glossary 

•  Record  Keeping 

-  Individual  Computer  Disk  for  Tracking  Individual  Data  for  SRS 

-  Bookmark  so  Learners  Can  Take  a  Break  and  Come  Back 

(This  is  particularly  important  for  those  with  short  timeslots) 

-  Record  of  Progress 

•  Screen  Design 

-  Colors  -  use  with  care  and  purpose 

-  Shapes  -  theme 

-  Visual  effect  of  Depth  and  Patterns 

-  Special  Effects  used  only  where  they  add  to  instructional  purpose, 

never  for  simple  glitz  effect 

-  Layout  and  Font  -  Readability 

•  Movement  -  Logical  Flow 

-  Learning  Objectives  Must  be  Developed  Prior  to  Content 

-  Path  must  create  logical  choices  and  allow  learner  control 

-  Transitions  are  Crucial 

-  Prerequisites  Identified  Early  in  Design  Phase 

•  Tram  Approach  -  Overlap  Duties 

-  Instructional  Designer 

-  Scriptwriter 

-  Graphic  Artist 

-  Videographer,  Standard  Production 

-  Videographer,  Special  Effects 

-  Programmer 

•  Planning  and  Preproduction  Tasks  Approximately  75%  of  Effort 

-  Module  1  Currently  in  Beta  Test  Phase 

-  Modules  2-6  in  various  stages  of  preproduction 

-  Different  from  linear  video    pay  attention  to  in  and  out  video/audio 

points,  video  &  audio  match,  light  levels,  sound  quality,  media  mix, 
animation  many  short  segments  that  must  fit  seamlessly 
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•  Some  Features 

-  Verbatim  Option  for  Those  Who  Wish  to  See  Words  or  Review  Audio 
.  Hot  Words 

•  Content  Map 

•  Workbook  Reference 

•  Experimental  Use  of  SVHS  Video  Master 

-  Sufficiently  good  quality 

-  Saves  money 

The  delivery  methodology  chosen  for  this  project  is  especially  appropriate  for  rural 
areas  because  it  can  be  used  in  local  offices  and  can  be  utilized  in  in-service  training  and 
instruction  for  groups  or  individuals.  The  delivery  format  of  the  training  modules,  a  self- 
contained  computer  based  multimedia  program,  uses  specific  skill  development  methodology 
and  takes  into  consideration  the  range  of  users'  skill  levels  with 
computers  and  supporting  technology.  The  ability  of  users  to  control  the  instructional 
process  and  to  record  their  progress  allows  remediation  when  necessary  or  the  option  to  move 
ahead  when  the  competency  is  reached.  Users  also  have  the  ability  to  learn  or  enhance  skills 
they  have  through  interaction  with  video  images  and  the  computer  program.  Program 
branching  allows  remediation  of  incorrect  responses  and  provides  for  choices  of  relevant 
information,  topics,  and  examples  for  students. 

Comprehensive  tests  assess  overall  understanding  and  skill  and  students  can  be 
reinstructed  in  those  areas  in  which  their  understanding  or  application  is  unacceptably  low. 
After  each  curriculum  component,  a  brief  review  determines  whether  the  student 
understands  the  component  well  enough  to  go  on  to  the  next  component.  Each  training 
module  is  concluded  with  a  test  on  which  a  user  must  achieve  a  passing  score  in  order  to 
receive  continuing  education  credit  (CEUs). 

Interactive  videodisc-based  instruction  has  the  advantages  of  timeliness,  flexible 
training  periods,  effectiveness,  and  multiple  applications.  It  is  capable  of  providing  the  type 
of  staff  development  needed  for  busy  rural  human  service  workers  who  must  fill  different 
roles  in  providing  services  to  their  clients  and  in  collaborating  effectively  with  educational 
and  other  service  professionals. 

There  are  a  number  of  characteristics  unique  to  the  rural  setting  that  make  training 
social  workers  in  rural  settings  a  challenging  endeavor.  As  is  true  with  many  other 
professionals  serving  rural  areas,  rural  social  workers  are  usually  generalists.  They  fill 
multiple  roles,  and  the  need  to  be  informed  is  vital  to  providing  comprehensive  services  to 
families. 

Establishing  expected  competencies  from  field  surveys  and  then  designing  video 
footage  and  computer  programs  to  teach  and  evaluate  are  major  steps  towards  providing 
essential  training  for  social  workers.  The  use  of  interactive  videodisc-based  instruction  holds 
promise  for  delivering  effective  training  that  is  individualized,  updatable,  and  accessible. 
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INTRODUCTION 


This  paper  reports  on  a  study  that  used  Digital  Video  Interactive  (DVI)  to  help 
students  learn  selected  concepts  of  physics.  The  purpose  of  the  study  was  to  investigate 
ways  in  which  new  video  technologies,  such  as  DVI,  can  be  used  to  help  students  learn 
concepts  of  physics  that  traditionally  have  been  difficult  to  teach.  By  making  those  concepts 
more  visual  in  nature,  the  topic  could  become  less  abstract  and  more  motivating  for  students 
to  learn. 

By  teaming  two  features,  the  concrete  visual  images  provided  by  video  together  with 
the  control  over  that  video,  which  can  be  obtained  from  digital  techniques,  we  hope  to  be 
able  to  make  the  abstract  concepts  of  modern  physics  more  understandable  to  students,  in 
this  way,  it  may  be  possible  to  attract  the  interest  of  students  who  traditionally  have  not 
chosen  to  pursue  the  study  of  physics.  This  approach  may  make  the  physics  of  the 
twentieth  century  more  attractive  to  all  students,  thus  making  positive  improvements 
toward  equity  within  science  education. 

The  theoretical  underpinnings  of  a  project  of  this  nature  must  build  from  research 
and  knowledge  about  how  students  learn.  Unfortunately,  very  little  research  has  been 
completed  on  student  learning  of  modern  physics.  Posner,  et  al.  (1982)  have  looked  at  the 
learning  of  relativity  as  part  of  a  study  on  conceptual  change.  Fischler  and  Lichtfeldt  (1989, 
1992)  have  tested  a  method  of  teaching  quantum  concepts  to  German  school  students.  Very 
little  research  has  been  reported  on  the  student  preconceptions  or  learning  in  these  specific 
topic  areas,  however  research  has  shown  that  interactive,  visual  technology  has  had  an 
impact  on  the  learning  process. 

Several  researchers  report  positive  effects  of  interactive,  visual  technology  on  student 
performance.  Mayton  (1991)  reports  that  students  who  use  interactive,  visually-based 
instructional  materials  outperform  others.  Abrams  and  Streit  (1986)  report  greater  learning 
gains.  Bunderson,  Olson,  and  Baillio  (1981)  report  faster  learning  and  better  retention 
among  students  with  access  to  such  material.  Finally,  Cushall  (1987)  reports  more  positive 
attitudes  r  nd  enthusiasm  for  the  material  when  students  use  technology  within  science 
classes. 

If  teachers  are  to  use  technology  effectively  in  their  teaching,  then  they  need 
meaningful  experiences  with  using  the  technology  as  learners  and  users  (Baird,  1991; 
Brooks  &  Kopp,  1989;  Smylie,  1989).  Future  science  teachers  need  experiences  using  the 
technology  during  the  course  of  their  training  (Sununu,  1986;  NSTA,  1990). 

Concepts  of  physics  are  often  presented  in  an  abstract  format  and  are  difficult  for 
students  to  understand.  By  making  the  instruction  more  visual,  we  hoped  to  aid  the 
students1  understanding  of  selected  concepts  in  beginning  physics.  This  project  used  digital 
video  interactive  (DVI)  to  produce  and  deliver  data  visualization  exercises  to  students  who 
were  studying  frames  of  reference. 

The  project  included  software  design  and  production  as  well  as  formative  and 
summative  evaluation.  This  paper  will  describe  the  development  and  formative  evaluation 

processes. 


o35 


BACKGROUND 


CD-ROM  and  videodisc  technology  can  provide  visuals  for  interactive  instru:tion  but 
they  offer  limited  visual  capability.  Each  can  provide  a  moving  image,  but  not  in  the  real- 
time sense  of  being  able  to  collect  and  display  experimental  data  as  it  is  collected.  DVI 
technology  has  a  wide  range  of  capabilities  and  shows  great  promise  for  a  variety  of 
educational  situations  (Ripley,  1990).  DVI  provides  real-time  motion;  students  can  collect 
and  visualize  data  as  well  as  interact  with  the  image  on  the  screen.  The  full  motion  image 
is  contained  to  a  window  on  part  of  the  screen  so  the  remainder  of  the  screen  can  be  used  for 
other  purposes,  such  as  graphing  data.  With  the  increased  emphasis  on  using  computer- 
based  multimedia  for  instruction,  there  is  a  need  for  evidence  of  its  potential  positive  effect 
on  learning. 

There  are  many  reasons  to  use  computer-based  multimedia  for  instruction;  the 
potential  for  visual  learning  and  interaction  are  two  of  those  reasons.  Research  provides 
ample  evidence  for  the  power  of  visuals  and  interactive  multimedia  to  enhance  learning 
(Carlson  &  Falk,  1990-91;  Dwyer,  1978,  1992;  Knupfer  &  Clark,  1992).  Computer-based 
activities  provide  opportunities  for  interaction  in  more  than  one  sense.  Interaction  can  take 
place  between  a  student  and  a  computer  as  well  as  between  students  who  are  engaged  in 
cooperative  group  work. 

Because  this  study  contained  a  large  number  of  females  who  were  minimally 
computer  literate,  it  was  important  to  investigate  the  merit  of  grouping  students  as  well  as 
possible  gender  and  attitude  considerations  prior  to  conducting  the  exercise.  A  review  of  the 
literature  indicated  that  students  benefit  from  working  in  heterogeneous  groups  as 
compared  to  working  alone  (Dalton,  Hannafm,  &  Hooper,  1989;  Gabbert,  Johnson,  & 
Johnson,  1986;  Hooper,  1992;  Johnson,  Skon,  &  Johnson,  1980;  Simsek  &  Hooper,  1992; 
Yager,  Johnson,  Johnson,  &  Snider,  1986).  In  addition,  heterogeneous  grouping  of  students 
is  preferable  to  homogeneous  grouping  (Knupfer,  1993). 

There  are  many  positive  benefits  to  cooperative,  group  work  including  the  ability  for 
students  to  observe,  imitate,  and  learn  from  each  other  (Rysavy  &  Sales,  1991).  Students 
learning  with  computer-based  multimedia  in  individualized  situations  control  the  pace  and 
sequence  of  instruction  while  those  learning  in  small,  cooperative  groups  express  more 
satisfaction  (Adams,  Carson,  &  Hamm,  1990;  Carlson,  1991).  Also,  cooperative  learning 
has  been  rated  particularly  appealing  to  low-ability  females  (Dalton,  et  al,  1989).  The 
combined  strategy  of  providing  computer-based  technology  and  cooperative  learning  is 
especially  positive1  for  female  students;  this  combination  has  revealed  improved  attitudes 
toward  computers  and  perhaps  even  more  important,  it  promotes  equalized  status  and 
respect  for  alt  group  members  instead  of  competitive  and  individualistic  learning  (Johnson, 
Johnson,  Stanno,  19HH).  Cooperative,  computer-based  learning  reveals  no  significant 
differences  in  the  performance  of  male  and  female  students  (Mevarech,  Stern,  &  Levita, 
H)87;  Webb,  1988). 

A  number  of  studies  have  been  conducted  to  compare  cooperative  and  individual 
learning  strategies.  According  to  Johnson  and  Johnson  (1989),  a  cooperative  learning 
strategy  allows  students  to  work  together  to  increase  performance  and  achieve  shared  goals; 
an  individual  learning  strategy  requires  students  to  work  by  themselves  to  accomplish  their 
own  goals.  Several  reviews  of  research  suggest  that  cooperative  learning  affects  students 
performance,  productivity,  transfer  of  learning,  time  on  tasks,  and  attitude  (Johnson  & 
Johnson,  1989;  Rysavy  &  Sales,  1991;  Sharan,  1980;  Slavin,  1990). 


Investigations  about  attitudes  toward  instruction  that  utilizes  computers  reveal 
mixed  results.  Some  studies  report  no  significant  effects  on  learner  attitude  (Robyler, 
Castine,  &  King,  1988;  Dalton  &  Hannafin,  1988),  but  the  majority  of  studies  indicate 
improved  attitudes  toward  computers  (Kulik  &  Kulik,  1987)  or  toward  the  subject  matter  at 
hand  (Menis,  Snyder  &  Ben-Kohav,  1980;  Kulik  &  Kulik,  1986). 

Keller's  (1987)  ARCS  model  suggests  that  motivation  in  an  instructional  setting 
consists  of  four  components:  attention,  relevance,  confidence,  and  satisfaction.  Finally,  Debe 
(1990)  suggests  that  computer-based  multimedia  best  can  be  utilized  in  education  as  a  tool 
to  help  structure  inquiry  based  on  higher-order  thinking.  Debe  draws  attention  to  the 
similarities  between  collaborative  interaction  with  peers  and  the  team-based  approaches 
underlying  today's  science. 

This  study  grouped  students  into  cooperative,  heterogenous  groups  for  a  DVI  exercise 
meant  to  stimulate  inquiry  and  promote  higher-order  thinking  to  clarify  a  difficult,  abstract 
physics  concept.  Students  engaged  in  an  activity  that  would  draw  their  attention  to  a 
relevant  task,  and  would  hopefully  build  confidence  and  satisfaction  through  a  successful 
experience  with  physics. 

Initially,  this  study  sought  ways  to  improve  the  teaching  of  modern  physics  from  two 
sources.  The  first  source  was  the  literature  related  to  the  teaching  of  modern  physics.  The 
second  source  was  the  people  who  work  within  the  field;  a  series  of  questions  were  directed 
to  the  physicists,  teachers,  and  students  who  are  subscribers  to  the  PHYS-L  computer  list. 

A  review  of  the  literature  revealed  a  dearth  of  information  about  ways  to  improve  the 
traditional  physics  curriculum.  Although  some  teachers  and  physicists  have  questioned 
which  ideas  of  modern  physics  should  be  considered  central  to  a  student's  development  and 
which  ideas  are  over  emphasized  in  the  traditional  curriculum,  those  questions  are  rather 
limited  in  both  number  and  scope.  Instead,  most  of  the  literature  on  teaching  modern 
physics  concentrates  on  "tricks  of  the  trade"  along  with  experiments  and  demonstrations 
that  are  most  appropriate  to  students  who  have  a  strong  interest  in  physics.  There  is  very 
little  written  about  shaping  the  content  and  and  delivery  of  the  physics  curriculum  to  bwomo 
more  appealing  to  students  who  lack  a  natural  interest  in  the  topic. 

The  series  of  questions  to  the  physicists,  teachers,  and  students  who  subscribe  to 
the  Phys-L  list,  seemed  to  offer  promise.  That  group  of  people  has  an  active  interest  in 
physics  and  views  technology  as  an  acceptable  means  of  addressing  the  problems  of 
teaching.  Unfortunately,  that  attempt  at  gathering  information  did  not  bring  any  useful 
results.  The  few  responses  that  directly  concerned  the  physics  curriculum  and  methodology 
seemed  to  suggest  a  "pet  topic"  rather  than  any  deep  thought.  This  type  of  response 
probably  resulted  from  either  a  lack  of  creative  thinking  beyond  the  traditional  formats  for 
teaching  modern  physics,  or  hasty  replies  to  e-mail  without  taking  time  to  reflect  on  the 
topic.  Perhaps  this  approach  could  be  tried  again  in  the  future,  following  some  discussion  of 
concrete  examples  about  how  DVI  could  be  use  in  teaching  physics. 

Some  examples  of  DVI  were  developed  and  presented  at  the  a  series  of  physics 
conferences  beginning  with  Winter  Meeting  of  the  Association  of  American  Physics  Teachers 
(AAPT).  People  were  able  to  interact  with  the  system  and  discuss  various  options  for 
improvement.  This  feedback  method  yielded  helpful  suggestions  concerning  focus  topics. 
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With  the  lack  of  response  to  the  electronic  questionnaire,  research  ideas  were 
developed  along  three  parallel  tracks  with  the  purpose  of  investigating  appropriate  teaching 
and  learning  strategies,  as  follows: 

*  Development  of  short  lessons  using  DVI  to  demonstrate  its  utility  to  the  physics 
teaching  community. 

*  Development  of  concepts  and  relations  among  concepts  in  modern  physics  for 

different  groups  of  students. 

*  Plans  for  testing  some  teaching  material  that  used  traditional  paper  as  the 

medium  but  with  an  untraditional  approach,  such  as  visual  quantum  mechanics, 
with  the  idea  of  converting  it  to  interactive  learning  materials  using  DVI. 

Each  path  would  converge  on  some  major  themes  in  the  future.  This  paper  describes 
the  project  in  its  current  state,  with  both  the  successes  and  problems  associated  with  using 
the  DVI  format.  It  addresses  DVI  demonstrations,  presentation  level  video,  student 
learning  of  modern  topics  in  physics,  nontraditional  approaches  of  the  content  of  modern 
physics,  DVI  as  an  experiment,  hardware  and  software  changes,  support  for  the  project,  and 
accomplishments  to  date. 

CONSIDERATIONS  ABOUT  DVI  MATERIALS 

The  project  considered  the  equipment,  its  capabilities,  the  curriculum,  and  the 
student  audience  as  follows: 

DVI  Hardware 

The  project  began  by  using  the  Action  Media  750  system  and  exercises  were 
constructed  about  both  visual  space-time  diagrams  and  variable  quantum  effects.  The  new 
generation  of  hardware,  the  Action  Media  II  system,  added  certain  capabilities,  but  at  the 
same  time  sacrificed  others.  The  Action  Media  II  system  was  used  to  construct  exercises 
about  physically  changing  reference  frames  as  well  as  development  of  mock-ups  of  progress 
on  the  750  System. 

Topics  in  Modern  Physics 

The  topics  selected  from  the  curriculum  of  the  beginning  level  physics  class  were 
visual  space-time  quantum  physics,  relative  motion,  and  Quantum  Physics. 

Real  Time  Video  Capture 

The  video  capturing  feature  allowed  students  to  conduct  a  physics  experiment, 
capture  a  video  of  the  experiment  on  fixed  disk,  and  then  collect  and  analyze  data  from  the 
captured  video.  Students  could  ask,  "What  if*  questions,  change  the  experiment 
appropriately,  and  immediately  analyze  the  data  from  the  new  experiment  by  following  its 
plot  on  a  computer-generated  graphic  display  of  the  data. 

Student  Audience 

The  level  of  the  students  was  a  major  consideration  in  the  materials  design  and  the 
topic  selection.  The  a  large  class  of  college  students  who  were  studying  teacher  education, 
were  enrolled  in  the  an  introductory  physics  course  that  included  this  curricular  material. 
These  students  were  not  science  majors,  but  education  majors  who  needed  to  study  science. 
The  project  was  visual  in  nature  and  promoted  visual  thinking,  so  it  carried  the  possibility 
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of  using  the  material  with  younger  students  of  high  school  age  or  upper  elementary  school 
age,  who  were  visual  thinkers. 

DIGITAL  VIDEO  INTERACTIVE  DEMONSTRATIONS 

The  ability  of  the  DVI  system  to  capture  video  in  real  time  and  manipulate  that 
data  was  of  central  interest.  This  feature,  called  Real-Time  Video  (RTV)  by  Intel,  allows 
individual  users  to  capture  video  to  the  hard  disk  at  30  frames  per  second.  Because  the 
video  information  is  stored  on  the  disk  in  digital  form,  one  may  collect  information  from  the 
images  by  performing  analysis  of  them  and  may  alter  them  in  some  simple  ways. 

The  first  attempt  at  using  RTV  resulted  in  gathering  distance-time  data  about  a 
moving  object.  Adhering  to  the  theme  of  investigating  topics  of  contemporary  interest,  a 
program  was  written  which  analyzes  the  motion  of  a  spherical  pendulum  which  can  undergo 
chaotic  motion.  To  accomplish  this,  a  video  camera  was  mounted  so  that  it  could  view  the 
end  of  the  pendulum  as  it  moved.  Then  the  video  signal  of  pendulum's  image  was  captured 
by  the  computer.  Upon  completion  of  the  video  capture,  an  image  analysis  program  found 
the  location  of  the  end  of  the  pendulum  and  recorded  its  coordinates  for  each  frame  of  the 
video.  With  this  information  available,  the  motion  video  segment  could  be  played  back 
while  simultaneously  displaying  a  graph  of  its  coordinate  points. 

One  limitation  of  the  DVI  is  it's  small  window  size,  yet  this  feature  also  has  the 
advantage  of  offering  the  opportunity  to  show  four  windows  on  the  screen  simultaneously. 
For  consistency  and  clarity  of  thought,  a  decision  was  made  to  present  consistent  types  of 
information  within  the  windows.  One  window  always  displayed  the  video  of  the  actual 
motion  while  the  other  windows  revealed  areas  where  the  students  could  view  plots  of 
distance,  velocity,  or  acceleration  versus  time,  or  distance  versus  velocity.  For  each  graph, 
the  points  were  plotted  as  the  pendulum  moved  to  the  appropriate  location  on  the  screen. 
Thus,  the  students  could  watch  two  dimensional  motion  which  they  created  and,  at  the 
same  time,  see  the  graphs  of  various  quantities. 

This  application  of  DVI  is  somewhat  similar  to  the  standard  approach  of  using 
range  finders  or  "smart"  pulleys  to  collect  data  from  one-dimensional  motion.  For  two- 
dixnensional  motion,  Luchner  and  Dengler  (1989),  Keshishoglow  and  Seligman  (1989),  and 
Huggins  (1988)  have  created  computer  interfaces  which  can  collect  information  from  one 
bright  spot  in  a  video  scene.  The  DVI  data  collection  scheme  has  both  advantages  and 
disadvantages  in  comparison  to  these  other  methods.  Unlike  the  standard  range  finder,  the 
DVI  and  other  video  methods  are  not  limited  to  one  dimensional  motion.  For  the  study  of 
chaotic  events  this  additional  dimension  is  very  important. 

DVI  has  an  advantage  over  the  other  video  methods  because  the  actual  video  scene 
is  recorded  on  the  fixed  disk,  Thus,  the  students  can  view  their  own  experiment  many  times 
while  they  look  at  different  ways  of  analyzing  it.  In  the  other  methods,  the  experiment  itself 
is  not  recorded  visually;  only  numbers  from  it  are  recorded.  Thus,  while  the  students  can 
complete  many  different  analyses  of  the  data,  they  cannot  look  at  the  experiment  exactly  as 
they  conducted  it, 

The  most  obvious  disadvantage  of  the  DVI  method  is  that  its  sample  rate  is  limited 
to  the  frame  rate  of  video  (30  frames  per  second);  range  finders  can  sample  the  data  at  a 
much  higher  rate.  Both  the  range  finder  and  the  older  video  methods  can  show  graphs  in 
real  time  as  the  data  are  collected.  With  DVI  the  computer  is  sufficiently  busy  collecting  and 
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digitizing  the  entire  video  image  at  30  frames  per  second  that  it  cannot  also  display  graphs 
;n  real  time.  Thus,  the  multistep  process  described  above  was  used.  The  lack  of  real-time 
graphical  display  does  not  appear  to  be  critical  to  this  type  of  measurement  because 
students  can  view  the  video  of  the  experiment  as  often  as  they  wish  and  compare  the  video 
to  the  graphical  data. 

A  second  application  of  DVI  turned  the  computer  into  a  system  for  recording  time- 
lapse  video.  Except  for  antiquated  Super-8  film  cameras,  the  equipment  for  time-lapse 
recording  is  very  expensive.  Thus,  this  capability  has  not  been  available  to  students  since  8- 
mm  film  essentially  disappeared  from  the  market.  The  DVI  system  is  programmed  to 
record  a  video  image  in  library  routines,  then  DVI  can  be  used  to  assemble  them  into  a  video 
sequence.  Thus  the  students  can  record,  playback,  and  analyze  events  which  result  in 
change  over  a  long  period  of  time.  For  example,  students  could  record  and  analyze  growth 
patterns  within  cells  or  crystals. 

RESEARCH  QUESTIONS 

The  goal  of  this  project  was  to  find  out  if  DVI  is  an  effective  way  to  visualize  data 
and  if  so,  whether  or  not  it  can  help  students  develop  an  understanding  of  the  frames  of 
reference  physics  concept. 

Formative  Evaluation  Questions 

1 .  How  difficult  was  it  for  the  students  to  understand  how  to  use  the  software? 

2.  How  difficult  was  it  for  students  to  actually  use  the  software? 

3 .  What  did  students  think  of  the  experience? 

4.  What  suggestions  did  students  have  for  improving  the  lesson  design? 

METHODOLOGY 

Subjects 

Subjects  in  this  study  included  two  classes  of  graduate  students  who  served  in  the 
formative  evaluation  by  reacting  to  the  exercise  as  it  was  developed  and  providing  a  basis 
for  revisions  to  improve  the  software  functionality  as  well  as  other  elements  of  the  lesson 
design. 

The  graduate  student  classes  were  of  two  types.  The  first  class  of  twelve  students 
who  tried  the  software  did  not  necessarily  have  familiarity  with  computers  or  instructional 
design.  These  students  were  given  extremely  little  introduction  about  what  to  expect  prior 
to  using  the  software,  but  had  a  follow-through  explanation  after  the  exercise.  They  used 
the  software  in  discovery  based  mode  an  a  test  to  nee  if  the  software  was  sufficiently  self- 
explanatory  to  allow  independent  learning.  Four  collaborative  groups  of  three  students  used 
the  software  and  were  allowed  to  discuss  the  physics  problem  at  hand  freely,  but  they  were 
instructed  to  keep  their  reaction*  about  the  software  confidential  until  the  data  were 
gathered. 

All  twelve  students  had  difficulty  with  both  using  the  computer  equipment  and 
understanding  the  physics  concept  of  frames  of  reference.  The  reaction  to  the  experience  was 
generally  negative  mid  reflected  frustration  despite  the  collaborative  structure  of  the 
exercise.  The  students  did  not  believe  that  this  exercise  could  stand  independently,  but 
suggested  that  the  instruction  include  an  introduction  to  both  the  equipment  and  the  frames 
of  reference  topic. 
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Based  upon  feedback  from  the  first  group  of  graduate  students,  modifications  were 
made  to  the  software  and  to  the  method  of  introducing  the  material  to  the  students.  The 
second  group  included  students  who  had  experience  with  design  for  interactive  multimedia- 
based  instruction.  This  class  was  given  slightly  more  introductory  information  about  both 
the  physics  frame  of  reference  concept  and  how  to  use  the  equipment. 

Modifications  to  the  software  functionality  included  an  improved  screen  design  with 
navigational  buttons  that  closely  resembled  a  video  cassette  recorder  (VCR)  to  be  used 
during  the  data  collection  and  analysis.  Functions  allowing  the  collection  and  analysis  of 
data  were  clearly  labeled  and  students  were  able  to  try  the  different  functions  at  will. 

Modifications  to  the  frames  of  reference  topic  did  not  delve  into  depth  on  the  concept, 
but  instead  provided  brief,  verbal  information  about  the  goal.  Students  received  this 
information  in  the  context  of  seeing  how  to  use  the  camera  to  record  a  dropping  ball  from  a 
stationary  cart  and  a  moving  moving  cart.  Like  the  first  group,  the  second  class  of  twelve 
graduate  students  tried  the  exercise  within  collaborative  learning  groups  of  three  and 
responded  to  the  experience  both  in  writing  and  verbally. 

This  group  of  students  reacted  more  favorably,  but  still  had  problems  with  understanding 
the  context  of  the  physics  lesson. 

The  frame  of  reference  lesson  was  intended  to  use  with  undergraduate  students  who 
were  taking  an  introductory  course  called  Concepts  of  Physics  in  partial  preparation  for 
careers  as  elementary  school  teachers.  Based  upon  the  feedback  from  the  formative 
evaluation,  the  lesson  for  the  undergraduate  students  included  the  frame  of  reference 
concept  within  the  first  few  lectures  of  the  semester.  This  preliminary  information  released 
the  students  from  the  content  struggle  and  helped  prepare  them  to  take  advantage  of 
extending  their  learning  to  a  different  level  of  experience. 

The  Concepts  of  Physics  class  utilized  lectures,  demonstrations,  and  laboratory 
exercises.  The  frame  of  reference  concept  was  taught  in  lecture  during  the  first  couple  of 
weeks  of  the  semester  as  part  of  the  regular  curriculum  and  thus  the  concept  was  tested  in 
the  first  exam  of  the  course.  Nothing  further  was  mentioned  about  frame  of  reference  during 
the  course.  During  the  last  two  weeks  of  the  semester,  the  students  used  the  DVI  exercise. 
The  exercise  consisted  of  a  pretest,  the  DVI  exercise,  and  a  posttest.  Finally,  four  frame  cf 
reference  questions  were  included  on  the  final  exam. 

Armed  with  the  prerequisite  physics  content,  the  students  were  then  grouped  into 
collaborative  groups  of  three  and  were  provided  with  more  information  about  working  the 
equipment  than  their  graduate  student  colleagues  had  been  given.  The  goal  was  to  provide 
enough  information  to  the  students  so  that  they  would  be  able  to  use  the  equipment  with 
comfort.  The  undergraduate  students  were  much  more  positive  about  the  experience  and 
nearly  100  percent  of  them  claimed  that  the  DVI  visualization  experience  helped  them  to 
understand  the  frame  of  reference  concept.  An  analysis  of  their  performance  scores  will  be 
provided  in  a  forthcoming  report. 

Although  the  exercise  provided  a  stimulus  to  engage  the  attention  of  the  first  two 
groups  of  graduate  students,  only  the  undergraduate  group  had  the  benefit  of  perceiving  the 
exercise  as  relevant  to  their  course,  confidence  in  knowing  their  goal,  and  satisfaction  of 
realizing  how  the  DVI  exercise  fit  within  the  whole.  Only  the  undergraduate  students 
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realized  the  four  components  of  Keller's  ARCs  model  (1987)  and  reacted  favorably  to  the 
exercise. 

RECOMMENDATIONS 

This  formative  evaluation  clearly  points  out  the  need  to  prepare  the  learners  wit?i 
adequate  information  about  the  topic  of  study  as  well  as  the  equipment.  Although 
developers  of  electronic  educational  material  might  view  the  use  of  such  material  as  easy, 
others  who  are  unfamiliar  with  the  material  are  likel  to  need  assistance  in  gaining 
perspective  on  the  material  prior  to  using  it. 

This  DVI  material  needs  to  proceed  to  summative  evaluation  to  measure  both  its 
perceived  value  and  it  effectiveness  for  helping  undergraduate  students  learn  about  the 
frame  of  reference  concept.  In  the  future,  it  might  be  possible  to  try  test  the  ability  of  the 
DVI  exercise  to  extend  the  physics  lesson  frame  of  reference  to  younger  learners,  such  a  those 
in  the  upper  elementary  school.  The  frame  of  reference  concept  has  traditionally  been  taught 
at  the  undergraduate  college  level,  but  the  rich  visualization  made  possible  with  DVI  might 
enhance  the  experience  to  the  point  where  it  becomes  valuable  for  younger  learners,  thus 
changing  the  science  curriculum. 
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ON  TEACHING  THEORY 


Last  year's  symposium,  as  has  been  the  case  in  many  of  the  other 
foundations  symposia,  was  concerned  with  the  theoretical  ideas  that  inform  our 
work.  This  year's  symposium  is  to  expand  on  the  theoretical  work  as  well  as  provide* 
some  indications  of  how  the  theoretical  work  is  carried  out,  i.e.,  what  is  the  nature  of 
our  practice.  This  is  not  to  be  misunderstood  as  a  "how  to"  session,  but  rather  to 
give  a  sense  of  the  struggles,  tensions,  joys  and  frustrations  of  our  work. 

What  I  have  to  share  here  is  not  magical  or  earth  shattering.  It  is  messy. 
This  messiness  is  not  rooted  in  a  lack  of  rigor,  but  rather  in  the  nature  of  the 
educational  experience  itself.  This  is  part  of  the  theoretical  framework  I  presented 
at  Inst  year's  symposium.  My  presentation  was  entitled  "Postmodern  Thinking  in  a 
Modernist  Cultural  Climate:  The  Need  for  an  Unquiet  Pedagogy". (1)  Unquiet 
pedagogy  is  a  term  from  Paulo  Freire.  Freire  states  that 

A  pedagogy  is  that  much  more  critical  and  radical  the  more  investigative  and 
less  eertninof  •certainties'  it  is.  The  more  unquiet  a  pedagogy,  the  more 
critical  it  will  become."  (Freire,  in  Kutz  &  Roskelly,  1991) 
Unquiet  comes  from  the  confrontation  between  one's  private,  personal  world 
ami  the  public  world.  We  always  find  ourselves  within  a  sociocultural  context.  An 
unquiet  pedagogy,  «  critical  pedagogy,  confronts  the  social-school -life- world  and 
acknowledges  the  cor#,v*  r.  part  of  the  human  condition.  An  unquiet/critical 
pedagogy  is  informed  by  and  allows  for  multiple  ways  of  knowing  and  experiencing 
the  world  (technical,  practical  and  emancipatory  forms  of  knowing). 

An  unquiet/critical  pedagogy  acknowledges  student/teacher  voice  in  the 
learning  process.  This  pedagogy  fosters  and  helps  create  democratic  forms  of 
schooling  through  confronting  issues  of  social  justice,  the  creation  of  community,  the 
common  good,  and  social  construction  of  knowledge. 

This  notion  of  an  unquiet/critical  pedagogy  as  presented  may  seem  to  be 
"reuioved"  from  experience,  too  abstract  and/or  nebulous.  I  think  that  is  because  of 
the  language  used  to  describe  it.  It  is  not  at  all  abstract  (for  me)  in  my  teaching.  I 
am  referring  specifically  here  to  what  I  teach:  1  teach  theory.  I  teach  a  course 
entitled  Sociocultural  Concerns  in  Education.  This  is  an  educational  foundations 
graduate  course  required  of  all  teacher  education  majors,  and  all  master's  programs 
within  the  department. 

What  1  try  to  represent  in  this  course  is  the  complexity  of  the 
teaching/learning  process,  the  sociocultural  nature  of  schooling.  I  represent  teaching 
and  schooling  within  a  political  context  (all  education  is  political),  and  that  teaching 
is  not  just  a  set  of  behaviors  but  an  orientation  to  the  world,  a  way  of  being  in  the 
world.  I  talk  about  teaching  being  a  moral  enterprise  because  we  intervene  in 
people's  lives. 

For  many  students,  to  talk  in  this  way  is  an  introduction  to  new  language,  to 
new  understandings  and  conceptualizations  of  teaching  (2).  Hence,  we  are  involved 
in  theory,  a  discussion  of  theoretical  positions  and  frameworks. 

I  do  not  look  for  concensus  in  my  class.  Students  do  not  have  to  agree  with 
mo.  1  do  look  for  a  willingness  to  engage  in  a  very  (at  times)  disturbing/unsettling 
process  of  self-reflective  study  of  teaching  and  schooling,  and  their/my  own 
perceptions  of  "how  thrnjjs  work".  "How  does  what  we  do  in  here  line  up  with  your 
perceptions  and  beliefs?"  "How  do  you,  as  a  scholnr/studcnt/rosenrchcr/  educator 
position  yourself  in  relation  to  this  information?"  "Does  this  information  cause  you  to 
re-think  your  position?" 
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Roger  Simon's  text  (1992)  Teaching  against  the  grain:  Texts  for  a  pedagogy  of 
possibility,  presented  a  challenging  way  for  me  to  reflect  on  my  teaching.  Simon 
suggests  that  students  fear  theory  classes,  and  because  of  this  fear  they  are  silenced 
(p.81  How  does  this  happen?  Many  students  do  not  want  to  appear  foolish  as  they 
try  to  enter  the  world  of  theoretical  discourse.  If  you  fear  ridicule,  you  fear  being 
ridiculous,  and  hence  students  say,  or  behave,  as  if  they  have  nothing  worthwhile  to 
say. 

Another  way  students  show  a  fear  of  theory  is  through  an  expression  of 
anger/frustration.  Simon  refers  to  this  as  the  "disruptive  character  of  theoretical 
knowledge"  (p.83).  This  is  experienced  not  only  during  class  discussion,  but  re- 
surfaces as  they  prepare  for  class,  do  assigned  reading,  etc.  In  the  class  discussions, 
it  is  represented  by  attacks  on  the  language  of  the  text,  the  language  used  to 
represent  theory.  This  language  is  referred  to  as  "jargon",  using  too  big  of  words, 
unintelligible  talk  or  writing.  Simon  interprets  this  language  as  adversarial.  It  is 
spoken  from  a  position  of  feeling  marginalized,  subjected,  and  dominated  (p.  83). 

I  sense  too  that  theoretical  discourse  can  be  very  alienating  to  students. 
They  already  have  a  common,  general  language/vocabulary,  as  well  as  seemingly 
common  experiences  that  are  easily  represented  by  that  vocabulary  and  language 
(their  taken-for-granted  ways  of  talking  about  and  understanding  their  everyday 
experiences).  To  be  expected  to  use  and  work  with  a  new  language  that  changes  the 
nature  of  that  experience  can  provide  a  challenge  to  your  thinking  that  can  be 
unsettling,  unquiet.  This  alienation  can  cause  a  sense  of  being  excluded  from  a 
discourse  that  is  supposed  to  make  sense  to  them,  supposed  to  have  some  relevance 
to  their  everyday  lives  (their  chosen  profession  of  teaching;  again,  the  disruptive 
character  of  theoretical  discourse,  theoretical  language).  (3) 

Another  manifestation  of  fear  of  theory  is  the  way  students  talk  when 
engaged  in  theoretical  discussion.  They  appear  to  be  hesitant,  tentative  (Simon, 
1992.  p.84). 

Often  times,  as  they  begin  their  response  they  say  "This  is  just  my  opinion,  hut...."  I 
believe  this  points  up  the  separation  between  subjective  and  objective  knowledge, 
and  which  is  more  valid.  This  also  suggests  the  major  separation  between  theory 
and  practice,  the  sense  that  "someone  else"  produces  the  theory,  and  as  teachers, 
they  just  want  to  know  how  to  use  it.  In  this  sense,  perspective  teachers  don't  do 
theoretical  work,  they  teach.  Theory  is  done  at  the  university,  teaching  is  done  in 
the  schools. 

This  new  language  also  tends  to  legitimate  a  particular  view  of  the  world 
that  is  not  familiar.  This  unfamiliar  language  suggests  that  their  way  might  be  less 
adequate.  Unless  a  real  open  forum  er:  ts  within  the  classroom,  the  pressure  to 
Kive  the  professor  what  s/he  wants  can  he  unquieting:  they  still  must  get  a  grade  for 
this  class  (Simon,  1992,  p.  89).  It  is  difficult  to  throw  off  all  those  years  of  schooling 
that  required  of  them  that  they  "Play  by  the  rules". 

Another  text  I  use  in  my  course  is  Mike  Rose's  Lives  on  (he  boundary:  The 
struggles  and  achievements  of  America's  underprepared  (1989).  This  text  helps  to 
understand  the  alienation  some  students  feel  as  they  enter  into  the  academic  world. 
For  those  of  us  who  have  been  in  academia  for  a  while,  we  can  forget  what  a  strange 
alienating  place  higher  education  can  be.  Mike  Rose  writes  of  education  being  an 
invitation  into  many  forms  of  discourse.  These  discourses  require  that  students  not 
only  learn  new  vocabulary,  but  understanding  the  language  systems  that  help  us 
engage  the  world  in  different  ways  (e.g.  the  language  of  the  sciences,  the  language  of 
philosophy,  sociology,  art,  literature,  etc). 


It  is  important  to  understand  that  I  am  not  talking  about  individual 
inadequacies  here,  I  am  not  talking  about  lazy  students.  I  am  not  blaming 
students,  nor  am  I  apologizing  for  my  teaching.  I  am  talking  about  human 
capacities,  individual  histories  and  the  social  form  of  the  institution.  And  1  am 
saying  that  study  is  intensely  hard  work.  Hence  I  cannot  just  look  at  theory  as  an 
academic  issue,  an  academic  question.  Simon  (1992)  strongly  states  that  theory  in 
education  has 

The  potential  consequences  for  the  organization  and 
disorganization  of  identities  make  a  pedagogical ly  motivated 
confrontation  with  theory  a  potentiallythreatening  and  noxious  situation. 
In  this  sense,classroom  language  practices  are  not  only  a  mode  of  social 
organization  but,  also  potentially,  a  mode  of  disorganization,  (p.  92) 
How  can  we  work  in  "theorizing  ways"  with  each  other?  To  return  to  Paulo 
Freire's  notion  of  an  unquiet  pedagogy,  a  critical  pedagogy,  we  can  teach  dialogically. 
The  preconditions  for  dialogue  to  take  place  are  demanding: 

1.  a  profound  love  of  humankind 

2.  humility 

3.  intensive  faith  in  people  (an  a  priori  faith 

in  the  individual 

4.  trust  (established  through  dialogue 

5.  hope  (rooted  in  our  sense  of  incompleteness, 

our  vision  of  what  could  be:  possibility 

6.  critical  thinking.  (Freire,  1970) 

Dialogue  is  always  rooted  in  our  lived  experiences  within  our  context.  This 
engagement  of  ourselves  with  each  other  and  our  world  is  most  difficult  as  well  as 
"messy".  We  spend  much  time  in  talking  and  listening  to  each  other  about  our 
understandings  of  the  world  (issues  under  discussion),  as  well  as  engage  in  academic 
readings  on  the  same  issues.  My  intent  here  is  to  have  our  conversation,  our 
dialogue,  grounded  in  our  experiences.  We  go  back  and  forth  between  the  theoretical 
literature  and  our  lived  experiences.  (See  Freire,  1970,  and  Simon,  1992). 

Students  in  my  class  arc  also  doing  something  else.  They  are  engaging 
others  outside  the  classroom  in  the  conversation  (See  Carson,  1986).  Students  are 
doing  field  research,  conducting  interviews,  reformulating  what  they  learn  through 
experience  as  well  as  through  their  study.  I  have  provided  a  rather  standard  model 
for  asking  questions,  problem -posing,  describing,  interpreting  and  evaluating  their 
data  from  nn  interview  process  (see  Research  Project  form,  below). 

What  is  critical  here  is  how  the  students  position  themselves  to  their  own 
studios,  their  experiences  in  light  of  their  studies,  and  the  experiences  of  others 
within  their  community.  How  do  we  construct  a  view  of  the  "world"  (issues)?  How  do 
we  view  ourselves  within  this  theoretical  discourse?  Can  we  understand  the 
arguments  of  our  experiences,  the  texts,  and  our  data? 

As  I  stated  in  the  beginning  of  this  brief  paper,  what  I  do  is  not  magical  or 
earth  shattering.  It  is  one  way  for  my  students  and  1  to  become  more  active  and 
critical  in  understanding  each  other  and  our  context.  It  allows  us  to  deal  with 
conflict.  They  tell  me  they  have  a  greater  sense  of  having  control  over  their  own 
learning.  Perhaps  this  will  provide  an  experience  where  they  are  less  alienated  from 
the  theoretical  discourse.  It  is  not  perfect,  and  it  is  still  messy. 


ENDNOTES 


1.  Paper  can  be  found  in  Proceedings  Selected  Research  Papers.  AECT/RTD 
1993  Conference,  New  Orleans,  LA.  This  is  a  brief  statement  of  my  position  in  the 
form  of  a  talking  paper.  An  expanded  and  more  clear  statement  of  my  position  can 
be  found  in  Koetting,  1994. 

2.  Examples  of  this  new  language  would  be  from  social  theory,  including 
issues  of  social  reproduction,  cultural  capital,  achievement  ideology,  the  politics  of 
education,  etc. 

3.  An  example  of  this  sense  of  alienation,  exclusion,  frustration  is  when 
some  students  are  first  introduced  to  ideas  in  social  theory,  they  express  a  sense  of 
"feeling  stupid11,  inadequate,  unable  to  understand.  They  complain  they  have  to 
constantly  look  words  up  in  the  dictionary;  they  have  always  felt  successful  in  school 
before,  but  in  this  class  they  feel  lost  and  "dumb". 
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RESEARCH  PROJECTS 

General  guidelines: 

1.  Identify  issue/problem  to  be  investigated.  You  must  define  and  describe 
the  problem/issue  in  narrative  form.  You  will  need  to  be  specific  here  in  defining 
terms,  establishing  meaning  right  from  the  Htart,  This  will  holp  in  the  next  phase, 
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which  is  to  develop  your  questions  that  will  solicit  information. 

2.  Develop  your  questions.  What  information  are  you  looking  for?  What 
should  be  the  nature  of  your  questions?  How  can  you  avoid  yes/no  questions?  You 
may  develop  this  section  in  narrative,  and  you  must  include  a  list  of  the  questions  in 
an  appendix.  Be  sure  to  get  demographic  data,  e.g.  male/female,  age,  schooling,  etc. 
You  should  consider  the  nature  of  your  questions  when  planning  for  the  demographic 
data  needed  to  interpret  findings  (#4). 

3.  Report  your  findings  (data).  What  did  the  people  say?  This  is  an 
objective  (i.e.  descriptive  as  opposed  to  interpretive  or  evaluative)  statement  of  your 
data.  Detail  is  important  here.  Let  the  people  speak  for  themselves,  and  report 
that  in  sufficient  "quantity"  that  the  reader  can  get  a  "feel"  for  the  conversations. 

4.  What  do  (does)  your  findings  (data)  mean?  How  do  you,  as 
researcher/student  scholar  situate  yourself  in  relation  to  your  findings?  Does  it  fit 
with  what  you  expected?  Does  it  match  or  give  additional  insight  into  what  you 
think/study  about?  Does  it  match  your  beliefs? 

5.  What  do  you  conclude  from  your  work?  What  are  the  implications  of  what 
you  just  learned? 

This  is  not  a  survey.  This  is  an  oral  interview,  a  conversation. 
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Abstract 


Hypertext  is  well-suited  for  educational  applications  where  open  learning  and 
knowledge  exploration  is  desired.  In  such  applications,  principles  of  good  hypertext  interface 
design  should  be  employed  to  avoid  navigational  problems  so  as  to  maximize  learning. 
Interface  design,  however,  may  also  directly  enhance  acquisition  of  a  particular  knowledge 
structure.  An  interface  that  is  based  upon  an  expert's  structural  knowledge  map  may  assist 
novice  learners  in  developing  a  more  sophisticated  knowledge  structure  more  closely 
resembling  that  of  an  expert.  Pathfinder  networks  have  been  used  successfully  to 
differentiate  expert  and  novice  knowledge  structures.  The  procedures  described  in  this 
paper  were  used  to  design  a  structured  interface  for  use  in  a  research  study  currently  in 
progress.  Two  microcomputer  programs  were  used  to  develop  the  interface;  The  Knowledge 
Network  Orientation  Tool  (KNOT)  to  create  the  pathfinder  network,  and  Asymetrix 
Toolbook  for  the  hypertext  application. 


Introduction 


Inherent  within  knowledge  is  a  network  structure,  or  schemata,  comprised  of  nodes 
(schema)  and  ordered,  labeled  relations  connecting  the  nodes.  Learning  occurs  when  new 
information  is  integrated  into  these  schemata  through  activation.  Due  to  the  limited 
capacity  of  working  memory,  most  network  nodes  will  be  inactive  at  any  given  time 
(Anderson,  1983).  Through  the  spread  of  activation,  inactive  nodes  become  active  in  working 
memory  an  nuw  material  provides  a  stimulus.  As  nodes  are  activated,  they  are  marked  so  a 
path  can  bo  retraced  to  the  starting  node.  Collins  and  Loftus  (1975)  claim  that  the 
act  ivation  of  nodes  in  the  network  decreases  proportionate  to  the  su^ngth  of  the  links, 
introducing  the  idea  of  Homantic  similarity  into  the  network.    Learners  generate 
elahoratiniiN  for  now  information  being  learned  (Haviland  &  Clark,  1974).  Elaboration 
iiHsint h  in  subsequent  retrieval  of  information,  which  is  structured  semantically.  The 
general  ion  and  modification  of  knowledge  structures  stored  as  mental  schema  is  a  critical 
component  of  learning  (Rumelhart  &  Ortony,  1977). 

Hypertext  computer  systems  are  similar  to  the  model  of  memory  described  above  in  that 
they  store  non  linear,  interrelated  units  of  information  associated  by  links.  While  learners 
benefit  from  the  ability  to  approach  a  domain  from  a  variety  of  perspectives  in  hypertext,  a 
persistent  problem  of  hypertext  systems  is  that  of  navigation.  When  too  many  choices  are 
available,  users  have  a  tendency  to  become  "lost"  within  the  network  (Conklin,  1987). 
Hypertext  interface  design  should  attempt  to  minimize  this  problem.  It  has  been  suggested 
that  effective  human-computer  interaction  depends  in  part  upon  communicating  system 
structure  and  organization  to  users  through  the  interface  (McDonald,  Dearholt,  Paap,  & 
Schvane veldt,  1986;  McDonald  &  Schvaneveldt,  1988).  The  authors  cited  above  assume 
users  will  develop  conceptual  models  of  a  system  that  can  be  characterized  as  schemata.  By 
providing  an  explicit  structure  in  the  interface  that  is  based  upon  a  structural  representation 
of  the  domain,  users  will  develop  conceptual  system  models  that  reflect  characteristics  of  the 
domain  being  studied  (McDonald,  Paap,  &  McDonald,  1990). 

Structural  Knowledge 

Although  research  has  suggested  that  graphical  browsers  in  general  do  not  facilitate 
the  acquisition  of  an  expert's  knowledge  structure  by  novices  (Jonassen  &  Wang,  1992),  we 
anticipate  that  by  using  a  specific  type  of  graphical  browser,  learners  may  modify  their  pre- 
existing schemas  (through  activation  and  elaboration)  differently  than  if  an  alternative  type 
of  browser  were  used.  By  using  a  pathfinder  network  as  the  browser,  learners  may  develop 
schemas  that  are  more  consistent  with  that  of  an  expert.   The  conceptual  model  that 
hypertext  users  develop  is  a  mediating  type  of  knowledge  termed  structural  knowledge. 
Designing  a  hypertext  interface  after  an  experts  knowledge  structure  as  represented  by  a 
Pathfinder  network  may  help  users  to  develop  a  more  effective  user  model,  assisting 
learners  in  acquiring  higher-order  procedural  knowledge. 

Maine  Structural  Knowledge 

The  distinction  between  declarative  and  procedural  ways  of  knowing  is  common  in 
cognitive  psychology  (Ryle,  1949;  Gagn_,  Yekovich,  &  Yekovich,  1993;  Jonassen,  Beissner  & 
Yacci,  1993).  Structural  knowledge  is  an  intermediate  knowledge  between  descriptive  and 
procedural  knowledge.  Structural  knowledge  describes  how  declarative  knowledge  is 
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interrelated.  In  addition  to  knowing  that,  learners  need  to  know  why  in  order  to  know  how. 
Thus,  structural  knowledge  mediates  the  translation  of  from  declarative  knowledge  to 
procedural  knowledge  by  showing  how  declarative  knowledge  is  interconnected  (Jonassen, 
Beissner,  &  Yacci,  1993).  Structural  knowledge  is  essential  to  the  development  and  use  of 
higher  order  procedural  knowledge  applied  within  any  content  domain. 

Structural  Differences  between  Expe  ts  and  Novices 

Research  has  suggested  that  experts  understand  at  a  deeper,  more;  abstract  level  than 
do  novices  (Chi,  Feltovich,  &  Glaser,  1981).  In  addition,  experts  have  »  difficult  time 
verbalizing  their  knowledge,  since  much  of  their  knowledge  has  become  automatic  (Cooke, 
1990).  Further,  experts  and  novices  differ  in  the  facts  and  rules  in  memory,  as  well  as  how 
those  facts  and  rules  are  organized  in  memory  (Chi,  Feltovich,  &  Glaser,  19H1;  Murphy  & 
Wright,  1984).  Experts  possess  a  different  knowledge  structure  than  novices,  and  know 
when  and  why  specific  facts  or  rules  should  be  applied  to  specific  cases.  Although 
representing  expert  knowledge  is  of  particular  interest  to  expert  systems  research,  other 
computer-based  instructional  models  such  as  hypertext  systems  may  benefit  from  explicitly 
providing  novices  with  an  explicit  expert  knowledge  structure. 

The  Relationship  Between  Structural  Knowledge  and  Hypertext  Interface  Design 

The  knowledge  representation  and  acquisition  models  discussed  above  have 
implications  for  hypertext-based  instruction.  The  limited  capacity  of  working  memory 
restricts  learning.  Formal  learning  usually  consists  of  information  presented  in  a  sequential 
fashion,  either  through  verbal  or  textual  presentation.  A  learner  therefore  must  break  down 
sequential  material  into  manageable  chunks  that  can  be  transferred  from  working  memory 
to  long-term  memory  In  addition,  the  goal  of  learning  is  the  tuning  and  refining  of  one's 
mental  knowledge  structure.  This  means  that  a  learner  must  add  chunks  of  information  in 
working  memory  to  long-term  memory  by  integrating  these  chunks  into  pre-existing 
schemata  through  spreading  activation.  As  noted  above,  experts  have  well-developed 
mental .«  'heinata  that  differ  from  those  of  novices.  We  anticipate  that  novices  may  benefit 
from  hypertext-based  instruction  that  incorporates  an  expert's  knowledge  structure  for  two 
reasons.  First  a  useful  conceptual  model  for  the  hypertext  system  is  provided  for  the  user. 
Second,  an  explicit  cognitive  structure  of  knowledge  domain  becomes  a  component  in  the 
spreading  activation  process.  If  a  link  can  be  made  between  the  user  model  and 
representation  of  the  domain,  novice  learners  should  benefit  from  incorporating  an  expert's 
knowledge  structure  into  a  hypertext  user  interface. 

Assessing  Structural  Knowledge:  Pathfinder  Networks 

A  variety  of  techniques  for  analyzing  and  representing  structural  knowledge  have  been 
identified  (Jonassen,  Beissner,  &  Yacci,  1993).  We  selected  Pathfinder  networks  as  our 
technique. 

Pathfinder  Associative  Networks 

Pathfinder  associative  networks  (PFNets)  are  configurations  in  which  concepts  are 
represented  in  a  node -link- node  format.  The  network  is  constructed  from  proximity  rntings 
of  all  possible  pairs  of  semantic  terms.  Each  link  between  PFNet  nodes  is  assigned  a  value 
representing  the  strength  of  the  relationship.  The  links  can  be  directed  or  undirected,  and 
are  calculated  based  upon  the  minimum  distance  between  concepts.  A  link  is  constructed  if 
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the  distance  between  concepts  is  greater  for  every  possible  path  than  for  the  paired  terms. 
PFNets  are  similar  to  MDS  networks,  except  that  PFNets  do  not  require  ratio  assumptions 
about  the  proximity  data,  and  are  not  bound  by  the  hierarchical  constraints  in  most  other 
clustering  techniques  (Dearholt  &  Schvaneveldt,  1990). 

Using  thg  Knowledge  Network  Organizing  Tool 

The  Knowledge  Orientation  Network  Tool  (KNOT)  is  a  series  of  computer  programs 
available  on  multiple  hardware  platforms  for  constructing,  analyzing,  and  assessing 
pathfinder  networks.  The  program  uses  an  algorithm  that  analyzes  a  proximity  data  file 
created  by  semantically  differentiating  all  possible  pairs  of  selected  concepts  or  terms.  The 
examples  that  follow  are  part  of  a  research  study  being  conducted  hy  one  of  the  authors  of 
this  paper.  For  that  study,  the  KNOT  microcomputer  program  was  used  in  conjunction  with 
Asymetrix  Toolbook  as  a  relatively  inexpensive  and  readily  available  means  of  providing  an 
explicit  structure  in  the  hypertext  interface.  Although  the  examples  here  use  the  Microsoft 
Windows  environment  on  the  IBM  PC,  the  procedures  demonstrated  here  can  be  applied  to 
the  Macintosh  environment  using  the  Macintosh  version  of  KNOT  and  an  authoring  tool 
such  as  HyperCard 

Constructing  expert  and  novice  maps.  Expert  and  novice  maps  are  both  constructed 
using  the  same  procedures.  A  number  of  individual  KNOT  program  modules  are  used  to 
construct  and  analyze  PFNets.  The  IBM  PC  version  of  the  software  contains  both  command 
lino  and  graphical  user  interface  (GUI)  versions.  The  command  line  version  was  used  in 
designing  the  structured  interface  discussed  here.  Constructing  a  PFNet  requires  at  least 
three  steps.  First,  the  terms  representing  the  domain  must  be  defined.  Second,  the  terms 
are  stored  in  an  ASCII  text  file  entitled  TERMS,  for  use  with  the  RATE  program.  Third,  the 
RATE  program  is  used  to  semantically  differentiate  all  possible  pairs  of  terms. 

Define  the  concepts  that  represent  the  domain.  Proximity  data  is  based  upon  the 
rater's  differentiation  of  paired  terms  representing  the  content  domain.  In  the  design  for 
this  study,  eighteen  terms  representing  the  major  categories  of  moral  philosophy  were  used. 

Rate  all  possible  pairs  of  terms.  Ratings  can  be  acquired  by  using  the  RATE  module 
within  KNOT,  which  will  randomly  present  the  rater  with  all  possible  pairs  of  terms.  A 
proximity  file  is  created  after  all  pairs  are  rated.  The  RATE  program  uses  a  9  point  scale  for 
differentiation.,  with  the  end  points  representing  a  high  degree  of  similarity  or  dissimilarity. 
Instructions  for  individual  raters  and  the  direction  of  the  ratings  (whether  1  equals  highly 
dissimilar  or  similar)  can  be  specified  and  written  to  an  ASCII  file  entitled  INSTRUCT.  For 
our  study  a  single  subject  matter  expert  was  consulted  to  rate  the  terms  used  to 
differentiate  18  concepts  describing  moral  philosophy. 

Use  the  Layout  function  to  generate  the  pathfinder  network.  Once  the  proximity  data 
file  has  been  created,  the  LAYOUT  program  module  within  KNOT  is  used  to  construct  a 
PFNet  from  the  proximity  data.  Figure  1  shows  the  PFNet  produced  by  the  proximity  data 
resulting  from  our  rater's  data.  In  this  example  the  following  parameters  were  used:  q=-l, 
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Assessing  differences  between  two  or  more  PFNets 


The  similarity  of  two  or  more  PKNetfl  can  be  assessed  using  the  NETSIM.EXE  program. 
NETSIM  generates  a  rclatt'diicHH  coefficient  ranging  from  0  (very  dissimilar)  to  1  (identical). 
NETSIM  cun  be  used  for  comparing  individual  PFNets  from  a  group  of  experts,  comparing 
export  and  novice  PFNftn,  or  for  comparing  PFNets  from  the  same  user  generated  before 
and  after  instruction.  Although  PFNets  can  be  assessed  visually  for  similarity,  the  NETSIM 
function  will  more  accurately  assess  the  differences  between  expert  and  novice  networks. 

Hypertext  Interface  Design  and  Knowledge  Acquisition 

Given  the  navigation  problem  users  usually  experience  with  hypertext,  the  structured 
interface  under  discussion  was  designed  as  a  single  Toolbook  page  with  a  recordfield  for 
individual  text  nodes  embedded  on  the  background  of  the  page.  The  18  nodes  comprising 
the  PFNet  act  as  buttons,  which  display  the  text  window  and  textual  information  specific  to 
each  individual  node. 

The  windowed  interface 

The  text  window  is  hidden  from  view  until  the  user  selects  an  option  from  the  menu 
interface.  Once  selected,  the  text  window  containing  information  specific  to  each  individual 
node  fills  approximately  fifty  percent  of  the  screen.  The  interface  remains  visible  behind  the 
window,  A  scroll  bar  allows  the  user  to  view  all  the  text  within  the  window.  In  order  to 
provide  an  explicit  knowledge  structure  in  the  interface  itself,  the  text  window  cannot  be 
resized,  and  multiple  windows  cannot  be  opened  simultaneously.  Links  between  nodes  are 
represented  as  "hot  words"  in  the  text,  which  are  surrounded  by  a  rectangular  box.  Users 
also  have  the  option  of  closing  the  window  to  once  again  make  the  entire  menu  interface 
visible.   Figure  2  shows  an  open  window  with  the  interface  in  the  background. 

SEE  FIGURE  2  APPENDIX  D 

Providing  users  with  an  explicit  structure 

Since  users  frequently  returned  to  the  interface  map,  the  underlying  knowledge 
structure  was  made  explicit.  As  users  interact  with  the  system,  the  concepts  of  moral 
philosophy  are  learned  within  the  context  of  the  knowledge  structure  provided  by  the 
interface.  Given  the  research  and  theory  under  girding  Pathfinder  networks,  we  predict  that 
a  learner's  PFNet  resulting  from  this  interface  will  differ  from  a  PFNet  generated  if  a 
different  interface  (such  as  alphabetical  listing  of  the  terms)  were  used.  Further  research  is 
needed  to  substantiate  this  assumption. 

Designing  a  hypertext  menu  interface  from  a  PFNet 

Designing  a  hypertext  menu  interface  with  an  authoring  system  such  as  Toolbook  can 
be  accomplished  in  a  variety  of  ways,  Once  a  list  of  terms  are  paired,  the  *.LO  file  can  be 
displayed 

on  the  screen  using  LAYOUT.EXE  and  sent  to  the  printer.  Consider  the  following  design 
options: 

Freehand  drawing.  Use  the  *.LO  printout  as  a  template  and  reproduce  its  main 
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features  such  as  number  and  position  of  nodes  links  in  Toolbook  (or  HyperCard)  with  as 
much  accuracy  as  possible. 

Cut  and  Paste   Configure  your  computer  to  run  KNOT  from  within  Windows  (this  is 
not  recommended  by  Interlink).  Once  the  *.LO  file  appears  on  the  screen,  reduce  the  screen 
to  an  active  window  (by  using  the  ALT  -  ENTER  keystroke).  Take  a  "snapshot"  of  the 
window  using  the  ALT  -  Print  Screen  keystroke.  Minimize  KNOT  on  the  desktop,  and  paste 
the  screen  print  into  a  draw  or  graphics  application.  Copy  and  Paste  the  image  into 
Toolbook. 

Optical  scanning.  Scan  the  *.LO  printout  to  a  file.  Depending  upon  the  file  format  used, 
an  import  filter  that  will  allow  the  file  to  be  imported  directly  into  the  active  book  may  be 
available  within  Toolbook. 

Conclusion 

The  advent  of  a  number  of  microcomputer  hypertext  authoring  systems  allows  designers 
to  create  a  variety  of  instructional  and  supplemental  materials.  If  knowledge  acquisition  is 
enhanced  through  the  use  of  Pathfinder  networks,  we  encourage  their  use  in  the  interface 
design.  The  procedures  described  in  this  paper  were  used  to  design  an  instructional  module 
to  research  the  effectiveness  of  PFNets  as  a  blueprint  for  interface  design  and  the  effects  of 
such  an  interface  upon  the  learner's  post-treatment  knowledge  structure.  Clearly,  more 
research  is  needed  to  determine  the  potential  uses  of  PFNets  with  hypertext  systems.  We 
have  presented  a  design  model  for  constructing  a  PFNet  interface.  We  encourage  readers  to 
experiment  using  PFNets  with  hypertext  in  other  domains. 
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Computers  are  an  unavoidable  fact  of  life.  They  are  used  to  gather,  store,  process 
and  distribute  information.  More  than  ever  before,  the  nation's  workers  are  required  to  use 
computers  to  conduct  business.  And  that  means  the  nation's  educators  must  address  the 
demand  both  to  teach  and  use  computers  in  the  classroom. 

According  to  the  U.S.  Census  Bureau's  report,  "Computer  Use  in  the  United  States, 
1989,"  more  than  75  million  Americans  reported  using  a  computer  in  some  way.  That  figure 
was  up  from  47  million  in  1984  (U.S.  Department  of  Labor,  1992-1993).  All  indications  are 
that  the  trend  toward  increased  computer  use  will  continue.  For  students  graduating  from 
post  secondary  institutions,  it  is  essential  to  have  skills  in  microcomputer  applications  in 
their  chosen  career  field. 

In  response  to  the  computer  age,  many  educators  have  embraced  computer 
technology  as  an  effective  teaching  tool.  TI     effectiveness  of  computers  as  instructional  tools 
has  been  documented  in  a  variety  of  disciplines  (Fleming,  1988;  Hague  &  Mason,  1986; 
Sadler,  1987;  Wiley,  1987).  Classroom  integration  of  computer  technology  is  necessary  to 
meet  the  future  demands  of  society  and  the  workplace. 

Curriculum  designers  in  post-secondary  education  seem  to  have  abandoned  the 
debate  over  whether  computers  should  be  used  in  the  classroom  and  have  turned  their  focus 
on  the  questions  of  when  and  how  computers  can  be  used  effectively  in  education  (Loyd  & 
Gressard,  1984).  But  before  computers  can  become  an  integrated  part  of  the  educational 
environment,  educators  need  to  know  what  computer  literacy  students  possess  before 
entering  the  classroom.  Educators  also  must  understand  what  attitudes  students  harbor 
toward  computers.  Understanding  students'  fears  and  anxieties  will  help  educators  craft 
instructional  methods  to  overcome  specific  negative  attitudes  (Arch  &  Cummins,  1989). 

Although  limited  research  has  been  done  to  identify  students*  computer  literacy 
levels,  less  effort  has  been  devoted  to  understanding  factors  affecting  students'  attitudes 
toward  computers  as  they  enter  post-secondary  educational  institutions.  An  assessment  of 
students1  computer  literacy  and  attitudes  would  help  educators  in  curriculum  planning  and 
in  developing  strategies  to  incorporate  computer  use  successfully  into  various  instructional 
settings.  Such  a  literacy  assessment  would  be  useful  particularly  for  educators  who  teach 
classes  using  computers  as  part  of  their  instructional  methodology.  Identifying  the  existing 
computer  literacy  of  first-year  students  will  help  teachers  effectively  plan  curricula  that 
would  strike  a  balance  between  computer  training  and  content  instruction. 

The  status  of  students'  computer  literacy  is  of  interest  to  many  educators,  including 
instructors  who  teach  introductory  computer  courses  and  to  instructors  who  teach  other 
subjects,  such  as  writing  and  mathematics,  in  a  computer  lub.  Duo  to  the  varying  levels  of 
computer  literacy  among  first-year  college  students,  instructors  who  teach  courses  with  a 
computer  component  often  are  faced  with  the  difficult  task  of  teaching  students  with  a  wide 
variety  of  ability  levels  (Harrington,  1990).  Although  the  computer  software  used  by 
students  in  many  college  courses  is  fairly  simple  ami  easy  to  use,  the  students  still  spend  a 
significant  portion  of  class  time  mustering  software.  The  computer  technology,  then,  could 
interfere  with,  or  detract  from,  students'  ability  to  locus  on  course  content. 

In  an  effort  to  learn  more  about  students'  computer  literacy  levels  and  attitudes 
about  computer  use,  a  study  was  performed  during  the  summer  of  1993  to  evaluate  the 
computer  literacy  of  first-year  students  entering  the  University  of  Wisconsin-Eau  Claire.  It  is 
hoped  educators  may  use  the  study  in  a  wide  variety  of  subject  areas  to  help  plan  classes 
and  adapt  teaching  methodologies  to  meet  students1  needs. 

Research  Questions 
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The  following  questions  guided  the  study: 

1.  How  long  have  students  been  using  computers  as  they  enter  the  university  as  first-year 
students? 

2.  What  kind  of  computer  experience,  if  any,  do  incoming  first-year  students  have?  (e.g.  word 
processing,  desk-top  publishing,  programming,  spreadsheets,  data  bases,  graphics 
packages) 

3.  Where  did  students  obtain  their  computer  experience?  (e,g.  school,  work,  home) 

4.  Is  there  a  relationship  between  student  experience  levels  and  attitudes  toward 
computers? 

5.  Is  there  a  relationship  between  student  major  and  computer  attitudes? 

6.  Is  there  a  relationship  between  student  gender  and  the  level  of  computer  literacy? 

7.  Is  there  a  relationship  between  student  gender  and  computer  attitudes? 

The  answers  to  these  research  questions  will  be  used  to  identify  the  existing 
computer  literacy  of  incoming  first-year  students  in  order  to  help  develop  curricula  that  is 
relevant  to  the  students1  needs  and  to  help  develop  instructional  methods  which  take  into 
consideration  students1  attitudes  toward  computers. 

Review  of  the  Literature 

A  number  of  studies  have  been  conducted  in  educational  settings  to  measure 
students1  computer  literacy,  which  refers  to  students1  knowledge  about  computers  and 
computer  operations  (Oates,  1983),  and  to  identify  how  computer  attitudes  affect  students1 
ability  to  use  computers  (Strickland,  1989).  In  a  study  at  the  University  of  Virginia,  Loyd 
and  Gressard  (1984)  found  that  levels  of  computer  experience  varied  greatly  in  the  students 
they  studied.  Their  study  indicated  that  students  with  more  computer  experience  were 
significantly  more  confident  in  their  use  of  computers  than  were  students  with  little 
computer  experience.  Researchers  viewed  increased  confidence  as  a  positive  factor  which 
influenced  students'  general  attitudes  toward  computer  use.  There  is  some  evidence, 
however,  that  general  attitudes  toward  computer  use  differ  by  gender,  with  women  being 
less  confident  of  their  abilities  and  distrusting  computers  more  than  their  male  counterparts 
(Lockheed,  1985), 

A  study  conducted  at  Purdue  University  (Sullivan,  1989)  also  measured  students' 
knowledge  of  and  previous  experience  with  computers.  Sullivan's  study  sought  to  gain  a 
systematic  knowledge  of  students'  computer  backgrounds.  Sullivan  found  that  although  half 
the  students  surveyed  had  some  previous  experience  with  computers,  many  did  not  use 
computers  regularly  and  had  no  motivation  for  using  computers  in  their  classes.  Sullivan 
suggested  that  more  complete  information  about  students1  computer  literacy  could  help 
educators  eliminate  misconceptions  about  students'  abilities  and  attitudes  and  develop 
curricula  that  more  closely  meets  the  needs  of  students. 

Since  1985,  students  in  freshmen  computer  literacy  courses  at  Bentley  College  have 
participated  in  a  study  regarding  their  p  re-college  computing  experience  (Harrington,  1990). 
The  annual  surveys  indicate  that  the  number  of  students  with  a  background  in 
programming  is  decreasing,  while  the  number  of  students  with  some  background  in 
productivity  software  is  increasing.  The  school  has  used  the  results  of  the  study  to  tailor  its 
computer  literacy  course  to  students'  backgrounds. 

In  addition  to  examining  how  students'  computer  experience  affects  current  learning, 
several  studies  have  been  done  to  measure  how  past  computer  experience  and  attitudes 
about  computer  use  may  be  linked.  Howard,  Murphy  and  Thomas  (1987)  found  that 
increased  computer  experience  reduced  students1  computer  anxiety.  Their  study  found  that 
students'  attitudes  toward  using  computers  could  be  improved  by  providing  students  with 
more  computer  knowledge  and  experience;  more  hours  of  computer  experience  at  the  college 
level  seemed  to  promote  higher  levels  of  confidence  and  improved  attitudes  among  students. 
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The  researchers  suggested  that  students  who  enter  introductory  courses  with  minimal 
experience  and  little  knowledge  about  computers  are  more  likely  to  suffer  from  high 
computer  anxiety;  therefore,  classes  that  include  computer  training  should  be  designed  with 
the  needs  of  these  potential  students  in  mind  (Howard,  Murphy,  &  Thomas,  1987). 

Lee  (1986)  also  studied  the  effects  of  computer  experience  on  computer  attitudes.  In 
a  study  at  the  University  of  North  Carolina  at  Charlotte,  Lee  found  that  past  computer 
experience  significantly  affected  performance  in  classroom  activities  in  which  students  used 
computers.  Lee  suggested  that  minimal  experience  with  computers  may  be  sufficient  to 
reduce  anxiety,  because  there  was  not  a  significant  difference  between  the  performance  of 
students  whose  computer  skills  were  classified  as  "low  experience"  or  "high  experience." 
Other  researchers  also  have  hypothesized  that  inexperience  and  unfamiliarity  with 
computers  can  cause  high  levels  of  anxiety  and  that  such  anxiety  can  interfere  with  students1 
academic  performance  (Hedl,  O'Neil,  &  Hansen,  1973;  Johnson  &  White,  1980;  Johnson  & 
Johnson,  1981). 

Attitude  also  \.as  found  to  be  linked  to  computer  use  in  a  study  conducted  at  a  large 
midwestern  university  (Arndt,  Clevenger,  &  Meiskey,  1985).  The  researchers  found  that  high 
levels  of  computer  use  were  associated  with  positive  attitudes  toward  computers.  Students 
with  greater  amounts  of  computer  experience  and  relaxed  attitudes  toward  computers  were 
found  to  be  less  likely  to  view  computers  as  threatening  (Arndt,  Clevenger,  &  Meiskey, 
1985). 

Gender  also  has  been  studied  as  a  factor  related  to  students'  computer  literacy  and 
computer  attitudes.  In  a  study  at  a  private  liberal  arts  college,  Arch  and  Cummins  (1989) 
found  that  males  tended  to  use  and  like  computers  more  than  did  females.  The  researchers 
found  that  females  tended  to  approach  computers  in  a  lab  less  frequently  than  males,  and 
females  had  a  more  negative  attitude  about  computers  and  their  ability  to  use  them. 
However,  structured  instruction  in  the  use  computers  helped  both  males  and  females 
achieve  a  more  positive  attitude  toward  computer  use  and  more  confidence  about  their 
computer  skills  (Arch  &  Cummins,  1989). 

Another  study  (Morahan-Martin,  Olinsky,  &  Schumacher,  1992),  which  was 
conducted  in  1989  and  1990  at  Bryant  College,  provided  evidence  that  gender  differences  are 
present  in  the  computer  experience,  skills  and  attitudes  of  incoming  college  freshman. 
Although  no  differences  were  found  in  the  amount  of  computer  experience,  males  had  more 
experience  and  skills  than  females  in  specific  types  of  computer  use,  particularly 
programming. 

Two  studies  conducted  with  students  in  New  Jersey  also  showed  a  gender  difference 
in  attitudes  toward  computers  (Wilder,  Mackie,  &  Cooper,  1985).  These  studies  found  that 
both  males  and  females  perceive  computer  use  as  being  more  appropriate  for  males.  The 
researchers  suggest  that  students'  attitudes  toward,  and  gender-related  perceptions  of, 
computers  are  affected  by  previous  school  experiences. 

Additional  studies  also  have  found  that  factors  such  as  gender,  age,  experience  and 
interest  in  computers  can  affect  students'  attitudes  toward  and  use  of  computers  (Lockheed, 
1985;  Smith,  1988)  The  findings  of  these  and  other  studies  suggest  that  additional  research 
needs  to  be  done  to  identify  factors  that  could  influence  students1  ability  to  use  computers.  A 
1982  literature  review  by  Lawton  and  Gershner  (1982)  found  few  empirical  studies  about 
issues  related  to  computer  literacy  and  attitudes,  further  suggesting  a  need  for  additional 
research  in  the  area  of  factors  affecting  computer  literacy  and  attitudes, 

Methodology 

The  survey  used  to  measure  first  year  students'  computer  literacy  and  to  study 
factors  affecting  their  attitudes  toward  computers  was  administered  June  through  August  of 
1993  at  the  University  of '  Visconsin-Kau  Claire.  The  University  of  Wisconsin-Eau  Claire  is  a 


liberal  arts  college  with  an  enrollment  of  approximately  10,000  students. 

The  questionnaires  were  distributed  via  student  orientation  packets  to  the 
university's  2,000  incoming  first-year  students.  A  written  questionnaire  was  considered  an 
appropriate  data  collection  instrument  because  it  permitted  a  large  number  of  people  to  be 
surveyed  in  a  brief  period  of  time.  Researchers  had  no  contact  with  students  and  students 
received  no  oral  explanation  of  the  study.  Written  instructions  included  with  the 
questionnaires  asked  students  to  return  completed  surveys  to  a  drop-off  box  located  at  the 
registration  site.  Responding  to  the  survey  were  444  students  (152  women  and  289  men). 

Designed  to  collect  information  from  students  regarding  computer  abilities  and 
attitudes,  the  questionnaire  asked  students  to  record  the  amount  and  type  of  their  previous 
computer  experience,  the  source  of  experience  and  their  attitudes  toward  using  computers. 
Demographic  data,  including  age,  gender  and  major  also  were  collected. 

Questionnaire  responses  were  tabulated  with  the  help  of  UW-Eau  Claire's 
Computing  and  Networking  Services  (CNS).  The  SPSS  computer  program  was  used  to 
perform  the  analysis  and  to  calculate  frequencies  for  each  question.  CNS  also  helped 
cross-tabulate  survey  data  on  experience,  age,  major  and  gender  with  computer  attitude 
questions  in  an  effort  to  understand  factors  influencing  both  student  computer  literacy  levels 
and  students'  attitudes  toward  using  computers  when  they  entered  the  university.  For  the 
purpose  of  the  cross-tabulation,  experience  was  separated  into  three  categories:  low-,  mid- 
and  high -experience  users.  The  categories  are  defined  as  having  0  to  12  months  of 
experience,  13  to  36  months  and  more  than  36  montiis,  respectively.  Because  there  was  not 
a  significant  rage  of  ages,  age  data  added  little  information  to  the  study. 

Computer  Literacy 

The  results  of  the  study  indicate  the  majority  of  students  surveyed  had  some  degree 
of  computer  literacy  when  they  entered  the  university.  Nearly  90  percent  (86.7)  of 
respondents  said  they  possessed  some  computer  experience.  More  than  70  percent  had  three 
or  more  years  of  experience.  Most  cited  word  processing  as  providing  their  chief  experience 
with  computers.  Approximately  66  percent  said  they  had  basic  word  processing  skills,  while 
33  percent  described  their  word  processing  skills  as  advanced. 

The  study  found  that  when  students  were  asked  to  assess  their  computer  literacy 
with  more  specific  skills,  such  as  graphic,  desktop  publishing  and  spreadsheet  programs, 
their  literacy  declined  sharply.  For  example,  47  percent  of  students  said  they  had  no 
experience  with  graphic  programs  and  nearly  75  percent  reported  less  than  a  year  of 
computer  graphic  experience.  Roughly  66  percent  said  they  had  no  desktop  publishing 
experience  and  85  percent  had  less  than  a  year  of  desktop  publishing  experience. 
Spreadsheet  program  experience  also  was  limited.  Forty-two  percent  reported  having  no 
experience  with  computer  spreadsheets  and  78  percent  said  they  had  less  than  a  year  of 
experience  with  spreadsheets.  Nearly  all  students  (97  percent)  listed  high  school  as  the 
source  of  their  computer  experience. 

Computer  Attitudes  —  Experience 

Cross-tabulating  survey  data  collected  about  students'  levels  of  computer  literacy 
with  responses  to  attitude  questions  revealed  that  increased  experience  was  not  equated 
with  higher  confidence  levels.  Overall,  greater  computer  literacy  did  not  positively  improve 
students'  attitudes  toward  using  computers.  Indeed,  it  seemed  that  the  old  adage 
"familiarity  breeds  contempt"  (or  at  least  mistrust)  applied.  Nearly  61  percent  of 
high-experience  users  (those  with  36  months  or  more  of  experience)  reported  strongly 
avoiding  computers,  compared  to  21  percent  of  students  with  no  computer  experience  and  24 
percent  with  less  than  a  year  of  experience.  According  to  the  questionnaire  categories,  72 
percent  (314)  of  students  were  classified  as  high-experience  users  (having  36  months  or  more 
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of  experience);  9.6  percent  (42)  as  mid-range  users  (having  13  to  36  months  experience);  and 
18.3  percent  (80)  as  low«experience  users  (having  0  to  12  months  experience). 

Confidence  levels  also  appeared  to  slide  as  experience  rates  climbed.  For  example, 
22.3  percent  of  high-experience  users  strongly  agreed  or  agreed  with  the  statement,  "1  feel 
confident  using  computers/1  compared  to  35.8  percent  of  mid-range  users  and  45.8  percent  of 
low-experience  users. 

High -experience  users  also  were  more  critical  of  their  computer  abilities.  Nearly  55 
percent  strongly  agreed  or  agreed  with  the  statement,  "I  am  no  good  at  using  computers," 
compared  to  30.9  percent  of  mid-range  users  and  29.2  percent  of  low-experience  users.  In 
addition,  relatively  few  high-experience  users  said  they  enjoy  using  computers  (8.6  percent 
strongly  agree  or  agree  with  the  statement,  HI  enjoy  using  computers,"  compared  to  2.4 
percent  of  mid-range  users  and  0  percent  of  low-experience  users. 

Computer  Attitudes  —  Major 

Intended  major  appears  to  play  no  significant  role  in  shaping  student  attitudes 
toward  computer  technology.  Students  of  all  ages  and  majors  indicated  a  desire  to  avoid 
using  computers.  For  example,  63~percent  of  science  majors,  56  percent  of  business  majors 
and  50  percent  of  humanities  majors  reported  avoiding  computer  use. 

Confidence  levels  were  highest  among  humanities  majors,  perhaps  because  these 
students  are  not  required  to  have  highly  developed  computer  skills.  Roughly  31  percent 
strongly  agreed  or  agreed  with  the,  "I  feel  confident  using  computers,"  statement. 
Approximately  23  percent  of  science  majors  and  20  percent  of  business  majors  responded 
positively  to  the  question.  Humanities  majors  also  reported  grater  enjoyment  regarding 
computer  use  (12  percent);  while  8.6  percent  of  business  majors  and  3.6  percent  of  science 
majors  agreed  or  strongly  agreed  that  they  enjoy  using  computers. 

Science  majors  were  most  critical  of  their  computer  abilities,  with  57  percent 
indicating  they  are  "not  good"  at  using  computers,  compared  to  51  percent  of  business 
majors  and  44  percent  of  humanities  majors. 

Computer  Attitudes  —  Gender 

Gender  difference  was  found  to  be  a  factor  in  students'  attitudes  toward  computers. 
Nearly  58  percent  of  women  reported  avoiding  using  computers  compared  to  49  percent  of 
men.  Female  students  expressed  less  confidence  in  their  computing  abilities;  23  percent  of 
women,  as  opposed  to  34  percent  of  men,  agreed  or  strongly  agreed  that  they  are  confident 
using  computers. 

(tender  appears  to  play  less  of  a  role  in  determining  students'  enjoyment  of  using 
computers  thim  does  choice  of  major.  Roughly  H  percent  of  women  and  8  percent  of  men 
strongly  agreed  or  agreed  that  they  en|oy  using  computers.  Women,  however,  were  more 
likely  (of)  percent)  to  describe  themselves  us  "not  good"  at  using  computers,  compared  to  41 
percent  of  men, 

Conclusions 

The  study  provides  useful  information  about  the  computer  literacy  and  attitudes  of 
students  entering  the  University  of  Wisconsin- Kan  Claire.  Approximately  90  percent  of 
respondents  have  some  computer  experience,  Nearly  70  percent  said  thty  had  used  word 
processing.  Such  news  should  he  encouraging  to  instructors  who  plan  to  use  computers  as 
part  of  their  instructional  methodology.  It  appears  reasonable1  to  expect  most  incoming 
students  to  have  some  level  of  computer  literacy,  particularly  in  the  area  of  word  processing. 

However,  a  lower  number  of  students  report  having  experience  with  graphic, 
spreadsheet  or  desktop  publishing  programs.  Instructors  who  plan  to  incorporate  these 
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computer  applications  into  their  classroom  likely  will  need  to  spend  more  time  on  computer 
training  for  students  or  identify  other  opportunities  for  students  to  improve  literacy  with 
classroom  specific  programs. 

The  response  to  the  attitude  questions  indicate  an  area  of  challenge  and  opportunity 
for  educators.  While  a  number  of  studies  indicate  that  students'  attitudes  improve  and 
anxieties  decrease  with  greater  computer  experience  (Lee,  1986;  Loyd  &  Gressarde,  1984),  it 
appears  more  needs  to  be  done  to  lessen  students'  fears  about  computer  use,  to  help  them 
build  confidence  in  their  abilities  and  to  see  the  connection  between  computer  use  and 
success  in  the  workplace. 

A  number  of  approaches  may  help  achieve  these  goals.  Mandatory  training  is  one 
method  which  could  be  used  to  increase  students'  computer  literacy  and  allow  them  to  build 
confidence.  Computer  courses  could  become  a  required  part  of  the  curriculum,  just  as  math 
and  English  are  mandatory  classes  for  most  students. 

A  second  approach  would  involve  developing  training  to  address  students'  anxiety 
levels.  For  example,  the  curriculum  for  a  publication  production  class  in  which  most  students 
were  familiar  with  desktop  publishing  could  be  taught  differently  than  a  class  in  which  few 
of  the  students  had  computer  experience  or  in  which  experience  levels  varied  widely.  In  a 
Kent  State  University  study  of  computer  anxiety  considerations  related  to  the  design  of 
introductory  computer  courses,  Howard,  Murphy,  and  Thomas,  1987)  found  that  initially 
segregating  low-experience  and  high -experience  users  during  the  training  process  had 
beneficial  outcomes  for  both  students  and  instructors.  For  example,  students  with  less 
computer  experience  could  receive  introductory  information  while  mo:e  advanced  students 
cc old  work  on  enrichment  exercises.  Another  option  would  be  to  allow  more  experienced 
students  to  serve  as  mentors  or  tutors  to  less  experienced  students. 

The  information  gathered  in  the  study  offers  some  indication  that  gender  differences 
do  affect  students'  attitudes  toward  computers.  An  understanding  of  the  differences  in  these 
attitudes  and  how  they  affect  literacy  rates  could  help  educators  make  better  educational 
choices  about  the  selection  of  teaching  methods  and  curricula.  In  a  study  of  sex  differences  in 
attitude  and  computer  use  among  college  students,  Arch  and  Cummins  (1989)  found  women 
were  more  likely  to  improve  their  computer  literacy  and  attitudes  if  they  were  required  to 
participate  in  a  structured,  classroom-integrated  approach  to  computer  use.  Such  evidence 
may  prompt  instructors  to  design  methodological  approaches  that  accommodate  the  learning 
needs  of  both  men  and  women,  giving  specific  attention  to  students  who  would  henefit  from 
more  structured  training. 

Other  studies  support  the  tenet  that  men  and  women  may  view  computers 
differently  and  may  require  varied  forms  of  training  in  order  to  be  successful  in  computer 
use.  A  Bryant  College  study  (Morahan*Martin,  Olinsky,  &  Schumacher,  1992)  found  that 
females  perceived  computer  skills  as  more  useful  for  their  careers  than  did  males,  but  males 
wore  more  willing  to  purchase  computers  than  were  women.  Such  evidence  of  gender 
differences  suggests  that  determining  the  reasons  behind  the  gender  differences  and 
developing  alternate  approaches  may  be  required  to  help  male  and  female  students  improve 
computer  literacy  and  attitudes  toward  computers. 

It  is  hoped  the  results  of  the  study  can  be  used  to  help  educators  conduct  the  task  of 
curriculum  planning  and  to  develop  courses  that  take  into  consideration  students'  attitudes 
and  computer  literacy.  Additional  research  is  recommended  to  further  identify  trends  and 
patterns  regarding  college  students'  computer  literacy  and  attitudes. 
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Abstract 

A  study  was  done  at  the  University  of  Wisconsin-Eau  Claire  in  April  1992  to  assess 
the  computer  skills  of  freshmen.  Information  about  students'  computer  skills  was  obtained 
through  the  use  of  a  questionnaire.  The  questionnaire  was  distributed  in  four  sections  of  a 
mass  media  writing  class  in  the  UW-Eau  Claire  Journalism  Department, 

Information  was  gathered  about  the  amount  and  type  of  students'  previous 
computer  experience,  the  source  of  the  experience,  and  their  attitudes  toward  using 
computers.  Demographic  information,  such  as  age,  sex,  and  major,  was  also  collected. 

The  study  found  that  98  percent  of  the  students  surveyed  possessed  some  computer 
experience,  with  80  percent  of  the  respondents  reporting  more  than  a  year  of  computer 
experience.  Ninety  percent  of  the  students  had  used  word  processing,  and  most  of  the 
students  (94  percent)  reported  gaining  computer  experience  in  high  school. 

Over  half  the  students  who  reported  a  high  level  of  computer  experience  (more  than 
one  year)  also  stated  that  they  look  forward  to  using  computers  and  that  they  have 
confidence  in  their  computer  abilities.  Students  with  less  experience  more  frequently 
reported  avoiding  computers  and  lacking  confidence  in  their  computer  abilities. 

The  findings  of  the  study  suggest  that  most  students  entering  the  university  will 
have  some  computer  experience,  with  word  processing  being  the  most  commonly  occurring 
computer  skill.  Increased  experience  seems  to  create  a  positive  attitude  toward  computer 
use  and  to  improve  students'  confidence  about  their  computer  skills. 

This  study  was  a  pilot  study  for  a  research  project  that  will  be  performed  later  in 

1992. 
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Abstract 


This  study  investigated  the  effects  of  interface  tools  and  learner  cognitive  styles  on 
performance  in  searches  for  information  within  a  hypermedia  database.  The 
subjects,  75  students  in  a  university  ESL  program,  were  blocked  for  field  dependence 
and  assigned  to  four  treatments  which  differed  by  the  search  tool  available.  The 
subjects  searched  EarthQuest.  a  social  studies  and  science  hypermedia  database, 
for  facts  to  answer  practice  and  posttest  questions  on  science  topics.  Field- 
independent  subjects  had  significantly  better  achievement  than  field- dependent 
subjects  in  both  the  index/find  and  map  treatments  but  not  in  the  browser  or  all 
tools  treatments.  The  active  engagement,  transfer  of  concepts  to  new  contexts,  and 
high  information  processing  demands  that  accompany  effective  use  of  the  index,  find, 
and  map  tools  may  explain  the  greater  achievement  by  field  independents.  The 
interface  design  and  instructional  use  of  hypermedia  databases  should  reflect  the 
range  of  cognitive  styles  of  users. 
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The  Effects  of  Search  Tool  and  Cognitive  Style 
on  Performance  in  Hypermedia  Database  Searches 

The  use  of  hypermedia  systems  for  learning  and  information  retrieval  has  grown 
very  rapidly  in  recent  years.  Hypermedia  cover  a  wide  range  of  applications,  from 
online  documentation  and  help  systems  to  authoring  tools  for  instruction  and 
learning  (Jonassen  &  Grabinger,  1990).  Some  educational  researchers  have  been 
intrigued  by  the  possibility  that  hypermedia  can  be  structured  to  reflect  the  semantic 
network  of  a  subject  matter  expert's  knowledge  and  then  used  for  instruction  to  help 
learners  acquire  not  only  knowledge  but  also  the  expert's  knowledge  structures 
(Jonassen  &  Wang,  1993).  Others  see  different  potentials  for  the  use  of  hypermedia 
in  education.  Duchastel  (1990)  argues  that  the  greatest  impact  will  be  for 
information  retrieval,  allowing  easy  access  to  the  vast  amount  of  information  that 
will  be  available  electronically  in  the  very  near  future.  In  explaining  his  position, 
Duchastel  stresses  what  he  considers  the  essential  characteristic  of  hypermedia: 
information  organized  in  a  network  that  provides  the  user  with  "the  capability  to 
quickly  access  additional  information  related  to  the  information  currently  under 
consideration"  (1990,  p.  136). 

However,  accessing  information  from  a  hypermedia  system  is  often  not  an  easy 
task,  especially  for  novices.  Users  may  encounter  a  number  of  common  problems 
(Hammond  and  Allinson,  1989).  They  may  get  lost  or  have  difficulty  gaining  an 
overview  or  finding  specific  information.  They  might  wander  without  an  orienting 
goal  or  strategy,  or  they  may  find  the  interface  tools  difficult  to  use.  Such  problems 
have  prompted  research  on  the  manner  in  which  users  interact  with  hypermedia. 

The  focus  of  some  research  has  been  on  the  effectiveness  of  different  tools  for 
searching  hypermedia  databases.  Jones  (1989)  investigated  the  use  of  menus 
embedded  in  the  text  for  browsing  compared  with  indexes  for  searching  encyclopedia 
articles  in  a  hypertext  system.  Both  success  on  finding  facts  and  score  on  an 
incidental  learning  posttest  indicated  no  advantage  of  one  tool  over  the  other.  Using 
a  hypermedia-based  city  guide,  Hammond  and  Allinson  (1989)  studied  exploratory 
and  directed  searches  under  five  tool  use  conditions:  hypertext  links  alone,  with 
maps  added,  with  an  index  added,  with  tours  added,  or  with  all  of  these  tools.  A 
posttest  over  factual  knowledge  gained  from  exploration  and  a  search  for  specific 
information  in  the  database  showed  no  significant  effect  for  the  tool  used  in  either 
task.  Wright  and  Lickorish  (1990)  had  learners  use  two  hypertext  databases,  one 
with  a  book-like  structure  and  the  other  with  a  hierarchical  structure,  which  they 
searched  by  either  a  map  or  an  index.  To  answer  questions,  users  had  to  compare 
information  from  different  sections  in  each  database.  The  results  showed  no 
significant  differences  for  the  two  search  tools. 

The  studies  reviewed  above  provided  learners  with  little  or  no  practice  using  the 
tools  before  engaging  in  search  tasks.  In  the  present  study,  learners  received  a  45- 
minute  hands-on  orientation  to  searching  a  database  and  a  30-minute  practice 
search  of  the  database.  Cronbach  and  Snow  (1977)  provide  theoretical  and 
experimental  support  for  their  recommendation  that  study  treatments  should  be 
long  enough  to  allow  learners  to  gain  experience  with  instructional  procedures. 

Another  factor  which  may  account  for  differences  in  information  retrieval  from 
hypermedia  is  the  individual  characteristics  of  users.  One  of  the  most  extensively 
researched  characteristics  is  field  dependence,  a  psychological  construct  identified 
and  elaborated  primarily  through  the  work  of  H.  A.  Witkin  and  his  colleagues 
(Goodenough,  1986).  The  field  dependence  construct  describes  differences  of  ability  in 
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perceptual  and  cognitive  problem  solving.  In  tasks  that  require  disembedding  parts 
of  a  field  from  the  overall  organization  of  the  field,  field-dependent  individuals  will 
tend  to  experience  the  parts  as  "fused/'  whereas  field-independent  individuals  tend 
to  experience  the  parts  as  distinct  from  the  field  as  an  organized  whole  (Witkin, 
Oltman,  Raskin,  &  Karp,  1971).  This  stylistic  tendency  in  perception  is  also 
displayed  in  intellectual  functions,  as  what  is  termed  cognitive  stvle.  An  ability  to 
disembed  simple  figures  from  complex  designs  is  reflected  in  an  ability  to  solve  a 
cognitive  problem  by  isolating  a  critical  element  and  using  it  in  a  different  context. 
Thus,  as  Witkin  and  colleagues  (1971)  explained,  a  field-independent  person  is 
capable  of  a  more  analytical  cognitive  functioning  than  a  field-dependent  person,  who 
uses  a  more  global  approach. 

The  few  studies  that  have  investigated  cognitive  style  as  a  factor  in  use  of 
hypermedia  systems  found  performance  differences  between  field  independents  and 
field  dependents.  Repman,  Rooze,  and  Weller  (1991)  presented  a  hypermedia  lesson 
on  computer  ethics  to  study  the  effects  of  advance  organizers  and  structural 
organizers  as  a  function  of  the  cognitive  style  of  junior  high  students.  Different  mixes 
of  a  map,  an  outline,  and  screen  titles  differentiated  four  treatments.  Although  field- 
independent  learners  outperformed  field-dependent  students  in  all  treatments,  the 
absence  of  any  benefit  from  advance  or  structural  organizers  on  the  field-dependent 
learners  was  contrary  to  what  the  investigators  expected. 

En-route  performance  differences  were  found  by  Stanton  and  Stammers  (1990) 
when  they  compared  field-independent  and  field-dependent  adults  using  hypertext 
training  modules.  Post-hoc  classification  of  the  users  by  the  ways  they  accessed  the 
hypertext  structure  indicated  that  the  more  field-dependent  users  tended  to  move 
from  details  to  main  points  in  the  modules.  The  authors  concluded  that  this 
"bottom-up*'  strategy  reflected  the  tendency  of  field-dependent  individuals  to  develop 
a  mental  model  through  hands-on  experience  rather  than  to  fill  in  a  model 
formulated  previous  to  the  interaction  as  would  be  expected  of  field-independent 
users. 

When  Jonassen  and  Wang  (1993)  investigated  different  methods  for  making 
structural  knowledge  of  hypertext  information  accessible  to  learners,  they  observed 
treatment  differences  by  cognitive  style.  A  browser  treatment  used  pop-up  windows 
to  identify  the  semantic  nature  of  each  hypertext  link.  A  semantic  selection 
treatment  required  the  learner  to  choose  the  semantic  type  from  a  list  at  each  link  in 
order  to  navigate  across  links.  A  control  treatment  offered  no  structural  information 
at  the  links.  On  a  posttest  recall  task,  field-independent  learners  did  better  in  the 
control  and  semantic  selection  treatments  and  worse  in  the  browser  treatment.  The 
authors  suggested  that  this  result  reflected  the  preference  of  field-independent 
learners  to  restructure  information  rather  than  accept  the  structure  provided  by 
materials. 

The  cognitive  style  studies  described  above  examined  the  exploratory  behavior 
and  related  learning  of  users.  In  contrast,  the  present  study  set  specific  information 
retrieval  tasks  and  focussed  on  search  task  performance. 

Th   purpose  of  the  present  study  was  to  investigate  the  effects  of  search  tool  and 
cognitive  style  on  performance  in  hypermedia  database  searches.  The  search  tool 
variable  had  four  levels:  browser,  index/find,  map,  and  all  tools.  The  cognitive  style 
variable  had  two  levels:  field  independent  and  field  dependent.  Performance  was 
measured  on  searches  in  (he  database  to  find  specific  information.  Besides  search 
achievement,  patterns  of  tool  use  were  of  interest  in  this  study.  Also,  attitudes  of 
the  learners  toward  the  tasks  were  obtained. 
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It  was  hypothesized  that  the  four  treatments  would  differentiate  between 
learners  with  analytical  (field  independent)  and  global  (field  dependent)  styles  of 
cognitive  processing.  Therefore,  interaction  of  the  treatment  search  tools  with  the 
cognitive  styles  of  the  learners  was  expected.  The  browser,  map,  and  all  tools 
treatments  were  expected  to  be  used  with  similar  effectiveness  by  field-independent 
and  field- dependent  learners.  However,  the  index  and  find  tools  were  expected  to  be 
more  advantageous  than  other  tools  for  rield -independent  learners  and  to  be  less 
useful  for  field-dependent  learners. 

This  hypothesis  is  based  on  the  comparatively  greater  benefit  for  field  dependent 
learners  when  material  is  encountered  in  a  structured  or  organized  manner  (Davis, 
1991).  The  browser  and  map  tools  would  provide  a  more  consistently  structured 
user-content  interface  than  would  the  index  and  find  tools.  Use  of  the  index  and  find 
tools  would  involve  disembedding  words  and  concepts  from  their  context  at  a  screen 
and  transferring  them  to  other  contexts  at  other  screens.  The  expectation  that  field- 
dependent  learners  would  not  perform  better  than  field-independent  learners  in  any 
treatment  reflects  the  results  of  cognitive  style  research,  which  Davis  (1991)  found 
provided  no  evidence  of  an  advantage  for  field-dependent  learners  except  when  the 
context  has  social  relevance. 


Method 

Subjects 

Subjects  were  75  adult  students  (40  males,  35  females)  enrolled  in  an  intensive 
preadmission  English  as  a  second  language  (ESL)  program  at  a  large  southwestern 
university.  These  students  came  from  more  than  a  dozen  different  countries.  Their 
reading  comprehension  level  in  English  was  two  semesters  or  less  below  American 
freshman  student  proficiency. 

Materials 

Materials  in  this  study  included  a  HyperCard  database,  a  print-based 
orientation  packet,  a  print-based  practice  packet,  and  the  Group  Embedded  Figures 
Test. 

The  content  material  for  the  study  was  a  HyperCard  database  called 
Earth  Quest  (Stevens  &  Smith,  1990).  This  published  hypermedia  program  is 
designed  for  secondary  social  studies  and  science  instruction.  The  content  is 
arranged  hierarchically  under  five  major  headings:  Earth  (an  earth  science  focus  on 
planets,  land,  water,  air,  and  life),  Journey  (a  historical  approach  to  invention  and 
politics);  Environment  (interrelationships  and  ecological  issues  for  land,  water,  air, 
and  life),  &  World  Tour  (a  geographical  overview  of  nations  by  continent). 
Information  is  presented  on  over  80  screens  by  text,  graphics,  animation,  and  sound. 
Many  of  these  screens  include  pop-up  windows  with  s   oiling  text.  EarthQuest 
provides  five  basic  tools  for  navigation  and  search  within  the  database.  Graphic/text 
buttons  on  each  screen  can  be  clicked  to  move  to  the  content  they  represent.  Browser 
icons  can  be  used  to  navigate  between  any  contiguous  screens  at  the  same  level  and 
up  or  down  the;  content  hierarchy.  An  index  lists  the  contents  of  the  database  as 
one-  or  twoword  topics  which  can  be  selected  to  reach  the  content  screens?  for  those 
topics.  A  find  feature  searches  the  database  for  typed  text  strings.  Maps  display  a 
hierarchical  arrangement  of  the  content  and  allow  searching  by  category, 


subcategory,  and  individual  screen  topic.  Each  of  these  five  tools  can  be  used  to 
reach  any  content  screen  in  the  database. 

EarthQuest  was  modified  for  three  of  the  four  treatments  in  this  study.  The 
browser  treatment  had  the  index,  find,  and  map  tools  disabled.  The  index/find 
treatment  had  the  browser  and  map  tools  disabled.  The  map  treatment  had  the 
browser,  index,  and  find  tools  disabled.  The  all  tools  treatment  used  an  unmodified 
version  of  EarthQuest. 

The  orientation  packet  contained  handout  directions  to  familiarize  learners  v  **h 
the  search  tools.  The  directions  explained  how  to  use  each  tool  to  conduct  search* 
of  the  database  to  find  specific  information. 

The  practice  packet  contained  handouts  giving  practice  search  directions  and 
questions.  Each  of  the  five  search  questions  asked  for  the  name  of  the  database 
screen  where  information  was  located  as  well  as  the  answer  found  there.  The  first 
three  questions  required  searches  for  isolated  facts.  The  other  two  were  cause/effect 
and  main  point/supporc  questions  that  each  required  information  from  two  screens. 
Some  of  the  search  questions  provided  hints  about  options  selection.  One  of  the  fact- 
finding questions  was,  "Living  things  are  found  only  in  a  very  thin  layer  of  the 
Earth's  crust.  How  thick  is  that  layer?"  The  cause/effect  question,  along  with 
directions  for  it,  was  "Land  can  move  and  change  shape,  such  as  to  make  mountains 
and  valleys.  What  are  two  causes  of  this  kind  of  change?  (Hint:  Play  movies.)  Get 
your  answers  from  two  screens." 

The  Group  Embedded  Figures  Test  (GEFT)  (Witkin,  Oltman,  Raskin,  &  Karp, 
1971)  was  used  to  indicate  the  cognitive  style  of  each  learner  in  the  study.  The 
GEFT  measures  the  ability  to  locate  a  simple  figure  within  a  larger  complex  figure 
which  has  been  designed  to  obscure  or  embed  the  simple  figure.  A  low  score  on  this 
timed  group-administered  test  indicates  perceptual  field  dependence,  whereas  a  high 
score  indicates  perceptual  field  independence.  In  broader  terms,  the  GEFT  assesses 
a  global  versus  analytical  dimension  of  cognitive  style. 

Procedures 

Instruction  took  place  during  two  75-mmute  class  sessions  for  each  of  six  classes 
within  a  five-day  period.  The  students  were  asked  to  volunteer  their  participation  in 
a  study  using  a  computer  program  in  the  ESL  computer  lab.  Their  incentive  was  a 
daily  class  activity  grade  for  each  session. 

The  GEFT  was  administered  to  the  participants  at  the  start  of  the  first  session. 
Each  learner  then  received  an  orientation  packet.  Using  an  overhead  computer 
screen  display  of  EarthQuest.  the  investigator  led  the  learners  through  the 
orientation  directions  while  the  learners  performed  searches  in  the  database  at  their 
individual  computers.  During  this  45-minute  orientation,  all  learners  used  an 
unmodified  version  of  EarthQuest. 

Before  the  second  session,  the  GEFT  scores  were  used  to  block  the  learners  for 
assignment  to  the  treatment  groups.  For  this  blocking,  the  learners  were  designated 
as  field  dependent  if  they  scored  from  0  to  9  on  the  GEFT  (n  =  31)  or  as  field 
independent  if  they  scored  from  10  to  18  on  the  GEFT  (n  =  44).  This  grouping  was 
based  on  the  himodal  shape  of  the  GEFT  score  distribution,  which  indicated  a 
dichotomy  of  the  learners.  Equal  numbers  of  learners  were  randomly  assigned  to 
each  treatment:  All  Tools,  Browser,  Index/Find,  and  Map, 

For  the  second  session,  the  learners  were  assigned  to  their  respective  treatment 
versions  of  EarthQuest.  During  the  30- minute  search  practice,  learners  used  the 
practice  packet  to  search  the  database  with  iheir  assigned  treatment  tools.  They 
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received  feedback  by  turning  to  the  following  page  in  the  handout,  where  the  answer 
was  given  above  the  next  question. 

After  the  practice,  a  posttest  was  administered.  The  learners  were  allotted  30 
minutes  to  search  EarthQuest  for  answers  to  the  posttest  questions  using  their 
assigned  tools.  At  the  end  of  the  second  session,  the  learners  completed  an  attitude 
questionnaire. 

Criterion  Measures 

The  dependent  variables  in  this  study  were  achievement  on  posttest  searches, 
tool  use  during  posttest  searches,  and  learner  attitudes. 

The  posttest  was  parallel  in  format  to  the  practice  searches  but  from  a  different 
content  area  in  the  database.  Each  answer  received  one  point  if  the  information  was 
correct  and  another  point  if  the  location  of  that  information  in  the  database  was 
correct,  to  give  a  total  of  14  possible  points.  Identification  of  the  location  increased 
the  probability  that  learners  would  actually  perform  searches  in  order  to  answer  the 
questions  rather  than  rely  on  their  prior  knowledge  of  the  content.  The  Cronbach 
alpha  reliability  coefficient  of  the  posttest  was  .66. 

Tool  use  during  the  posttest  searches  was  recorded  for  each  learner  by  means  of 
a  tracking  script  added  to  the  EarthQuest  program.  This  script  captured  data  on 
the  tool  the  learner  selected  to  navigate  to  each  screen  and  the  time  when  each 
screen  was  accessed.  These  data  were  compiled  to  give  both  the  number  of  screens 
accessed  using  each  tool  and  the  time  spent  at  those  screens.  The  four  measures  for 
patterns  of  tool  use  were  number  of  screens  accessed  using  treatment  tools,  number 
of  screens  accessed  using  graphic/text  buttons,  time  at  screens  accessed  using 
treatment  tools,  and  time  at  screens  accessed  using  graphic/text  buttons. 

Attitudinal  data  was  obtained  by  a  4-point  LikerMype  questionnaire  consisting 
of  12  items.  The  questions  asked  learners  to  respond  to  statements  about  the  value 
of  the  lesson  and  their  enjoyment  of  it,  the  ease  of  use  of  the  database,  and  their 
understanding  of  the  directions  and  questions.  Two  questions  specific  to  each 
treatment  asked  about  the  usefulness  and  ease  of  use  of  the  tools  assigned  for  thai 
treatment.  The  alpha  reliability  of  the  questionnaire  was  .85. 

Design  and  Pata  Analysis 

A  4  X  2  (search  tool  X  cognitive  style)  posttest-only  experimental  design  was 
used,  with  random  assignment  to  treatment  groups  after  blocking  by  cognitive  style. 

Due  to  unequal  cell  sizes  in  the  study  design,  homogeneity  of  variance  for  the 
posttest  scores  was  a  concern.  The  Bartlett-Box  F  test  (Glass  &  Hopkins,  1984) 
indicated  that  the  scores  were  homogeneous  in  variance  among  cell  groups  (y  >  .25). 

An  alpha  of  .05  was  used  for  all  statistical  tests.  Posttest  achievement  was 
analyzed  with  ANOVA;  tool  use  and  attitudinal  data  were  analyzed  with  MANOVA. 
If  multivariate  significance  was  found  on  analysis  of  the  tool  use  and  attitudinal 
data,  univariate  tests  were  then  performed.  For  any  significant  main  effects  found 
for  the  search  tool  variable,  Tukey  multiple  comparisons  followed. 

Results 

Achievement 

Table  1  gives  mean  posttest  scores  by  level  of  search  tool  and  cognitive  style. 
The  mean  posttest  scores  by  level  of  search  tool  were  7.37  for  the  all  tools  group, 
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7.39  for  the  browser  group,  8.16  for  the  index/find  group,  and  8.32  for  the  map 
group.  The  mean  scores  by  level  of  cognitive  style  were  6.35  for  the  field  dependent 
group  and  8.84  for  the  field  independent  group. 

Table  1 

Search  Test  Scores  bv  Search  Tool  and  Cognitive  Stvle 


Search  Tool 


Cognitive  Style 
Total 

All 

Browser 

Index/Find 

Map 

Field  Dependent 

M 

7.56 

6.88 

5.14 

5.43 

6.35 

SD. 

2.83 

3.18 

3.29 

3.36 

3.16 

Field  Independent 

M 

7.20 

7.80 

9.92 

10.00 

8.84 

SD 

3.99 

2.82 

2.61 

1.59 

3.00 

Total 

M 

7.37 

7.39 

8.16 

8.32 

7.83 

£11 

3.40 

2.93 

3.fi« 

3.23 

3.28 

ANOVA  revealed  a  significant  interaction  hot  woo  n  search  tool  and  cognitive 
style,  E(3,67)  =  3.48,  n  <  .05,  ES  =  .37,  This  interaction  is  illustrated  in  Figure  1.  A 
simple  main  effects  test  (Ferguson  &  Takano,  1989)  suggested  that  within  the 
index/find  treatment,  fieldindependent  learners  Ql=  9.92)  performed  significantly 
better  than  field-dependent  learners  (M=  5.14).  £(1,67)  =  12.27,  %  <  .001,  ]££  =  .70); 
within  the  map  treatment,  fieldindependent  learners  (H  =  10.00)  also  performed 
significantly  better  than  field-dependent  learners  (M  =  5.43),  £(1,67)  -  11.74,  £  < 
.001,  ££  =  .68.  No  other  significant  differences  were  found  when  simple  main  effects 
tests  were  conducted. 
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Figure  1.  Effects  of  search  tool  and  cognitive  style  on  search  test  achievement. 
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ANOVA  also  indicated  that  cognitive  style  was  significantly  related  to 
achievement.  F(l,67)  =  12.65,  u  <  .001,  E£  =  .41.  Field-independent  learners 
performed  significantly  hetter  on  the  posttest  than  field-dependent  learners. 
ANOVA  did  not  reveal  a  significant  main  effect  for  search  tool  treatment. 


Tool  Use 

MANOVA  revealed  that  search  tool  treatment  had  a  significant  effect  on  tool 
use,  £(12,108)  =  3.85,  p.  <  .001.  Univariate  analyses  indicated  that  differences 
occurred  for  the  number  of  screens  accessed  using  treatment  tools,  Z(3,67)  =  8.21 ,  £ 
<  .001,  ££  =  .57,  and  for  the  number  of  screens  accessed  using  graphic/text  buttons, 
£(3,67)  =  4.68,  a  <  .01.  J£&  =  .43. 

Tukey  multiple  comparison  tests  suggested  that  learners  in  the  browser 
treatment  (M  =  32.37)  significantly  accessed  more  screens  using  their  treatment  tool 
than  did  learners  in  the  index/find  treatment  (M  =  8.57)  and  in  the  all  tools 
treatment  (M  =  14.33).  Tukey  multiple  comparison  tests  also  indicated  that 
learners  in  the  browser  treatment  (M  =  38.12)  significantly  accessed  more  screens 
using  the  graphic/text  buttons  than  did  learners  in  the  index/find  treatment  (M  - 
14.86). 

MANOVA  also  revealed  that  cognitive  style  was  significantly  related  to  tool  use 
£(4,51)  =  3.1 1,  fi  <  .05.  Univariate  analyses  indicated  that  differences  occurred  for 
the  number  of  screens  accessed  using  treatment  tools,  £(1,67)  =  14.13,  n  <  .001,  ££ 
=  .43.  Fieldindependent  learners  (M  =  27  H6)  significantly  accessed  more  screens 
than  did  fielddependent  learners  (M  =  12.93). 


Learner  Attitudes 

Three  of  the  75  learners  were  not  included  in  the  analysis  of 
questionnaire  data  due  to  incomplete  responses  or  unreturned  forms. 

Data  on  learner  attitudes  were  measured  on  a  1  to  4  scale  with  1  as  "strongly 
agree"  and  4  as  "strongly  disagree."  Learner  attitude  results  indicated  a  general 
liking  for  both  the  program  and  the  tasks,  with  a  mean  response  of  2.16.  The 
highest  response  (M  =  1.70)  was  to  the  statement,  "EarthQuest  is  a  good  program." 
The  lowest  response  (M  =  2.65)  was  to  the  statement,  "I  had  enough  time  to  answer 
the  questions." 

MANOVA  revealed  that  cognitive  style  had  a  significant  relationship  with 
attitude,  E(12,43)  =  2.60,  £  <  .05.  Univariate  analyses  indicated  that  significant 
differences  occurred  for  two  of  the  12  items.  Field-independent  learners  (M  =  1-93) 
responded  more  positively  than  the  field-dependent  learners  (M  =  2.33)  to  the 
statement  that  the  assigned  treatment  tool  <;was  useful  for  finding  the  answers  to 
the  questions,"  £(1,64)  =  5.77,  £  <  .05,  E£  =  .28.  However,  field-dependent  learners 
(M  =  1-44)  were  more  positive  than  the  field-independent  learners  to  the  statement, 
"EarthQuest  is  a  good  program,"  (M  =  189),  £(1,64)  =  4.55,  £  <  .05,  ES  =  .25. 

Discussion 

The  purpose  of  this  study  was  to  investigate  the  effects  of  search  tool  and 
cognitive  style  on  performance  in  hypermedia  database  searches.  Learners  assigned 
to  one  of  four  interface  tool  treatments  searched  for  specific  information  in  the 
EarthQuest  database.  Cognitive  style  was  defined  in  terms  of  field  dependence,  as 
measured  by  the  GEFT.  The  results  of  this  study  support  the  hypothesis  for  an 
interaction  of  search  tool  with  cognitive  style. 

As  expected,  the  search  achievement  in  the  index/find  treatment  was 
significantly  better  for  the  field-independent  learners  than  for  the  field-dependent 
learners.  This  superiority  may  be  a  reflection  of ''more  active  approaches  to  learning, 
such  as  hypothesis  testing  and  employment  of  verbal  mediators"  (Goodenough, 
1976,  p.  676),  approaches  which  a  variety  of  studies  have  associated  with  field 
independence  (Witkin,  Moore,  Goodenough,  &  Cox,  1977).  In  order  to  navigate  from 
screen  to  screen  in  EarthQuest  when  using  the  index  and  find  tools,  the  learners  had 
to  physically  select  a  word  or  phrase  by  either  typing  it  into  the  find  dialog  box  or 
choosing  it  at  the  index.  When  the  screen  accessed  by  this  action  was  in  a  different 
section  of  the  database,  the  learners  encountered  a  new  context.  It  therefore 
appears  likely  that  active  approaches,  such  as  hypothesis  testing  or  the  use  of  verbal 
mediators,  were  important  for  searches  employing  the  index  and  find  tools. 
Fieldindependent  learners,  being  more  inclined  to  engage  in  hypothesis  testing  or  to 
use  verbal  mediators,  would  be  expected  to  have  an  advantage  when  using  the  index 
and  find  tools. 

The  search  achievement  results  also  confirmed  the  expectation  of  no  significant 
difference  by  cognitive  style  in  the  browser  and  all  tools  treatments.  Cognitive  style 
research  offers  an  explanation  for  this.  Field-dependent  individuals  tend  to  use 
"more  intuitive  spectator  approaches"  to  learning  (Goodenough,  1976,  p.  67G).  When 
searches  in  EarthQuest  were  performed  using  the  browser  tool,  a  click  of  the  browser 
icon  accessed  a  contiguous  screen  at  the  same  level t  higher,  or  lower  in  the  content 
hierarchy.  Often  there  was  no  need  to  enter  a  different  section  of  the  database. 
This  less  active  style  of  interaction  would  benefit  field-dependent  learners. 
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The  similarity  of  the  all  tools  treatment  results  to  those  of  the  browser 
treatment  for  search  achievement  can  be  understood  from  the  patterns  of  tool 
selection  revealed  by  the  tracking  data.  When  the  all  tools  learners  accessed  screens 
by  treatment  tool,  70%  of  the  time  they  used  the  browser,  compared  with  28%  for 
the  index  and  find  tools  and  2%  for  the  map.  Since  the  all  tools  learners 
predominantly  used  the  browser,  it  is  not  surprising  that  their  performance  was 
similar  to  that  of  the  browser  learners. 

Search  achievement  results  for  the  map  treatment  were  contrary  to  expectation. 
The  field-independent  learners  assigned  to  use  the  map  tool  outperformed  the  field- 
dependent  learners  under  this  treatment  with  a  difference  and  an  effect  size  close  to 
those  found  for  the  index/find  treatment.  The  hypothesis  had  predicted  search 
achievement  of  the  map  users  similar  to  that  of  the  browser  users.  It  was  assumed 
that  using  the  maps  would  provide  the  learners  with  a  structured  interface  that 
could  accommodate  the  more  global  approach  of  field-dependent  learners.  However, 
unlike  the  other  tools,  the  maps  were  not  available  on  every  screen.  The  learners 
had  to  access  them  on  separate  map  screens.  To  move  through  the  map  hierarchy 
from  main  sections  to  subsections  in  order  to  access  individual  screens,  at  least  one 
additional  map  screen  had  to  be  selected.  The  map  screens  thereby  took  the 
learners  out  of  the  context  of  a  content  screen.  Field-independent  learners,  with 
their  greater  ability  to  transfer  concepts  to  new  contexts,  appear  to  have  an 
advantage  at  this  kind  of  map  interface. 

Additional  support  for  these  explanations  comes  from  information  processing 
research  on  cognitive  style  differences.  In  a  review  of  these  studies,  Davis  (1991) 
concluded  that  little  or  no  difference  was  found  between  field -independent  and  field- 
dependent  learners  when  a  limited  amount  of  information  was  processed.  However, 
field-independent  learners  were  consistently  more  efficient  in  situations  with  higher 
information-processing  demands.  Reflecting  Marchionini's  observation  that  "it  takes 
less  cognitive  load  to  browse  than  it  does  to  plan  and  conduct  an  analytical, 
optimized  search"  (1987,  p.  70),  in  the  present  study  it  appeared  that  the  browser 
tool  placed  less  information  processing  demands  on  learners  than  the  other  tools. 

The  results  for  patterns  of  tool  use  revealed  that  while  using  their  treatment 
tools,  the  field  independents  accessed  more  screens  than  did  the  field  dependents. 
This  adds  support  to  the  suggestion  that  the  field  independent  learners  were  more 
actively  engaged  when  searching  the  database  than  were  the  field-dependent 
learners. 

The  attitudinal  results  indicate  that  EarthQuest  was  generally  well-received, 
although  cognitive  style  differences  appeared  in  the  attitude  questionnaire 
responses.  The  fielddependent  learners  strongly  agreed  that  EarthQuest  was  a  good 
program;  the  fieldindependent  learners  were  less  positive,  hut  also  agreed  that  it 
was  good.  Locus  of  control  may  be  an  important  factor  in  this  difference  in  attitude. 
Field-independent  learners  tend  to  be  more  internally  motivated  and  less  influenced 
by  external  goals,  compared  with  fielddependent  learners  (Witkin  et  al„  1977). 
Locus  of  control  could  account  for  differences  in  attitude  found  by  Small  and 
Grabowski  (1992)  in  student  use  of  a  hypermedia  system.  Although  all  of  the  field- 
dependent  subjects  in  that  study  were  satisfied  with  only  the  hypermedia  lessons, 
most  of  the  more  field -independent  subjects  said  they  would  probably  consult  other 
sources  as  well.  In  the  present  study,  the  field-independent  learners  may  have 
considered  the  EarthQuest  lesson  an  inadequate  way  to  achieve  their  language 
learning  goals,  whereas  the  field-dependent  learners  appeared  to  be  more  accepting 
of  the  lesson  as  part  of  their  classwork  in  English. 


377 


The  results  of  tins  study  have  implications  for  the  design  of  hypermedia  and  its 
use  in  thr  chiKsvmun    Decisions  about  which  interface  tools  to  provide  for  database 
search  and  informal  ion  i  H  rieviil  should  take  into  account  the  cognitive  styles  of 
users.  Tools  such  as  a  browser,  which  can  be  used  effectively  without  placing  high 
information  profession  demands  on  the  user,  should  be  made  available  for  field- 
dependent  individuals.  Index,  text-string  find,  and  other  tools  that  allow  more 
analytical  and  opt)  mi  ml  searches  would  provide  fieldindependent  users  with  an 
efficient  method  for  information  retrieval.  In  a  variety  of  settings,  students  with 
different  cognitive  styles  <:nuld  learn  more,  and  perhaps  be  better  motivated,  if 
activities  involving  information  retrieval  accommodated  the  cognitive  processing 
approaches  of  t  hose  students. 

An  avenue  of  future  research  on  cognitive  style  factors  in  the  use  of  hypermedia 
could  involve  the  formulation  of  criteria  for  interface  tool  design.   Hutchins,  llollan, 
and  Norman  (1986)  use  the  term  direct  engagement  to  identify  the  nature  of  user 
interaction  with  computers.  When  there  is  greater  direct  engagement,  the  computer 
interface  becomes  more  transparent  and  the  user  more  involved  in  the  illusion  of 
acting  directly  upon  the  objects  in  the  task  domain.  Hutchins  and  colleagues  suggest 
that  this  feeling  of  directness  is  "inversely  proportional  to  the  amount  of  cognitive 
effort  it  takes  to  manipulate  and  evaluate  a  system"  (1986,  p.  95).  Perhaps  a 
measure  of  direct  engagement  could  he  applied  to  the  tools  used  in  hypermedia 
systems.  A  tool  which  provided  greater  direct  engagement  would  lessen  the  cognitive* 
effort,  or  information  processing  demands,  of  the  learner  and  would  especially  benefit 
those  with  a  more  field-dependent  cognitive  style. 

The  current  growth  in  the  use  of  hypermedia  both  in  education  and 
communications  suggests  that  it  deserves  the  attention  of  educational  technologists. 
Jonassen  and  Grabinger  (1990)  envision  the  possibility  of  an  extensive  impact  of 
hypermedia  on  our  lives  that  would  require  a  new  form  of  literacy:  "If  large  amounts 
of  our  reading  in  the  future  will  be  by  unguided  and  unconstrained  electronic  text, 
new  strategies  (a  hypermedia  literacy)  will  be  needed"  (p.  21).  If  our  work  as 
educational  technologists  is  to  include  facilitating  the  use  of  new  information  forms, 
we  need  to  consider  how  cognitive  style  and  other  individual  characteristics  affect  the 
performance  of  learners. 
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It  is  an  often  stated  conviction  that  producing  transfer  is  the  main  job  of  education. 
Yet,  an  increasing  body  of  research  shows  that  the  way  knowledge  is  presented  to  students 
in  school  and  the  kinds  of  operations  they  are  asked  to  perform  often  result  in  students 
knowing  something  but  failing  to  use  it  when  relevant.  Brown,  Collins,  and  Duguid  (1989) 
have  concluded  that  this  condition,  variously  referred  to  as  a  transfer  problem  or  the 
problem  of  inert  knowledge,  occurs  because  classroom  activities  lack  the  contextual  features 
of  real-life  problem-solving  situations. 

Today,  there  is  widespread  interest  in  learning  through  authentic*  use  as  the  theory  base 
for  situated  learning  matures  and  as  innovations  in  computer-based  multi-media  systems 
outstrip  development  of  theory-based  instructional  strategies  (Dick,  1991).  Increasingly, 
theorists  and  educators  are  promoting  reality-centered  projects,  theme-based  learning,  and 
other  kinds  of  activities  situated  in  real-life  and  life-like  contexts  as  ways  to  engage  students 
in  meaningful  learning  (Blumenfeld,  Soloway,  Marx,  Krajcik,  Guzdial,  &  Palincsar,  1991; 
Clinchy,  1989;  Wager,  1994).  At  the  same  time,  the  availability  of  powerful  low-cost 
computers  has  stimulated  interest  in  the  design  and  development  of  simulations. 
Simulations  have  long  been  used  to  deliver  instruction  in  educational,  military,  and 
industrial  settings  on  the  basis  that  they  increase  the  ability  of  participants  to  apply  what 
they  have  learned  in  the  classroom  to  the  real-world  or  transfer  situation. 

In  this  paper,  we  discuss  the  implications  of  authentic  activity  considered  as  a  model  for 
appropriate  learning  activity,  particularly  in  the  design  of  computer-based  simulations  and 
project-based  learning  activities.  We  suggest  that  there  is  much  more  to  transforming  the 
conventional  classroom  into  an  authentic  learning  environment  than  incorporating  features 
of  real-life  situations  into  school  work.  The  provision  of  computer-based  simulations  and 
reality-centered  projects  does  not  insure  that  a  student  will  assume  a  positive  orientation  to 
learning  nor  derive  the  benefits  of  in-context  learning.  Much  additional  support  is  required 
to  strengthen  the  tendency  of  the  learner  to  engage  in  intentional  learning  processes  and  to 
help  the  learner  progressively  assume  responsibility  for  learning. 

The  following  discussion  is  divided  into  three  sections,  with  each  section  organized 
around  one  of  three  questions  and  each  question  serving  as  a  heading  for  one  of  the  sections: 

(a)  What  are  the  characteristics  of  real  life  problem-solving  situations,  and  what  is  different 
about  problem  solving  in  school? 

(b)  What  are  the  characteristics  of  educational  simulations,  and  what  do  people  learn  from 
them? 

(c)  What  are  the  characteristics  of  an  authentic  learning  activity, 

and  how  is  it  related  to  problem-solving  in  real  life  and  simulated  situations? 

Question  1.  What  are  the  characteristics  of  real-life  problem-solving  situations,  and  what 
is  different  about  problem  solving  in  school? 

In  the  cognitive  apprenticeship  framework,  understanding  develops  through  application 
and  manipulation  of  knowledge  within  the  context  of  the  ordinary  practices  of  the  target 
culture--in  other  words,  through  authentic  activity.  This  principle  represents  the  primary 
rationale  for  using  authentic  activity  as  the  model  for  appropriate  learning  activities.  As 
Brown  et  al.  (1989)  suggested,  conventional  classroom  tasks  frequently  lack  the  contextual 
features  that  support  transfer  from  the  school  setting  to  the  outside  world.  For  advocates 
of  situated  approaches  to  learning,  the  provision  of  authentic  activity  in  schools  is  a  way  to 
increase  cognitive  engagement,  support  meaningful  learning,  and  facilitate  transfer. 
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In  this  section,  we  list  characteristics  of  real-life  problem-solving  tasks  that  are 
relevant  to  the  design  of  authentic  learning  environments: 

•  Conditions  are  frequently  ill-structured  and  problems  are  ill-formulated.  Hence,  as  Spiro, 
Feltovich,  Jacobson,  &'Coulson,  (1991)  suggested,  understanding  develops  through 
experience  in  multiple  case  contexts  and  from  multiple  perspectives  within  the  same  context. 

•  Practice  using  skills  is  embedded  in  performing  the  activities  that  justify  developing  the 
skills  in  the  first  place  (Brophy  &  Alleman,  1991). 

•  The  reasons  for  learning  something  or  performing  an  activity  are  clear.  Individuals 
assume  responsibility  for  establishing  and  monitoring  their  goals  and  strategies  when  the 
reasons  for  performing  procedures,  even  tedious  ones,  are  understood  within  the  context  of  a 
broad  global  task  environment  (Honebein,  Duffy,  and  Fishman,  1994). 

•  Projects  frequently  have  depth,  complexity,  and  duration  (Berliner,  1992).  People  have 
opportunities  to  engage  in  active  and  generative  problem-solving  activities  that  involve 
personal  values  and  beliefs.  As  a  result,  they  experience  a  feeling  of  ownership  over  the 
activity  and  its  goals,  and  thus,  the  tendency  to  engage  in  intentional  and  self- regulated 
learning  processes  is  enhanced. 

•  People  work  together  in  situations  where  the  intelligence  to  solve  a  problem  or  perform  an 
activity  is  distributed  across  a  group  of  peers,  a  learner-mentor  system,  and/or  an  electronic 
performance  support  tool  (EPSS)  or  other  form  of  cognitive  technology  (Pea,  1993).  The 
quality  of  interactions  between  participants  is  frequently  of  primary  importance  in 
undertaking  a  project  or  accomplishing  a  goal. 

•  People  work  on  solving  problems  that  do  not  already  have  known  solutions.  When  people 
work  collaboratively  on  solving  real-life  problems,  they  share  in  substantive  conversation, 
which  has  a  different  quality  from  conventional  school  talk  (Newmann,  1991).  An 
individual's  orientation  toward  learning  is  qualitatively  different  when  learning  is  embedded 
in  the  context  of  achieving  personally  relevant  and  valued  goals  versus  working  for  a  grade  or 
some  goal  that  is  far  off  in  the  future. 

Question  2.  What  are  the  characteristics  of  an  educational  simulation,  and  what  do  people 
learn  from  them? 

The  theoretical  assumptions  underlying  the  design  of  simulations  are  varied,  as  are  the 
purposes  for  which  they  are  used  and  the  contexts  in  which  they  appear.  According  to 
Cunningham  (1984),  a  simulatior  duplicates  some  essential  aspect  of  reality  for  purposes  of 
experimentation,  prediction,  evaluation,  or  learning.  An  educational  simulation  is  designed 
to  increase  one's  ability  to  respond  appropriately  in  a  real-world  or  transfer  setting. 
Participants  practice  decision-making,  problem  solving,  and/or  role  playing  in  the  context  of 
a  controlled  representation  of  a  real  situation  (Smith,  1986). 

From  an  instructional  design  perspective,  educational  simulations  support 
predetermined  learning  outcomes  by  providing  participants  with  opportunities  to  deal  with 
the  const  ^uences  of  their  actions  and  to  respond  to  feedback.  Within  Pea's  (1985) 
framework  of  distributed  intelligence,  computer-assisted  simulations  have  the  potential  to 
reorganize  mental  processes  by  "closing  of  the  temporal  gaps  between  thought  and  action 
[and]  between  hypothesis  and  experiment"  (p.  85).  Pea  has  proposed  that  by  allowing  the 
user  to  engage  in  "what-if  thinking"  through  a  partnership  between  user  and  technology, 
deep  qualitative  effects  are  made  possible  on  how  problem  solving  occurs. 

In  reviewing  the  literature  on  simulation,  the  definition  of  fidelity  appears  to  vary 
depending  on  the  context  to  which  it     applied  and  the  theoretical  orientation  of  the  author. 
For  example,  in  proposing  a  model  for  assessing  the  fidelity  of  task  simulators  used  in 
industry,  Bruce  (1987)  proposed  three  criteria:  (a)  physical  similarity,  (b)  functional 
similarity,  and  (c)  task  communality.  The  fidelity  of  a  simulator  within  Bruce's  fidelity 
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verification  model  is  determined  by  assigning  a  value  to  each  of  these  categories  and 
combining  them  to  produce  a  fidelity  index  for  a  particular  training  device.  In  contrast, 
Smith  (1986)  believes  that  the  essential  reality  factor  in  a  simulation  is  not  the  form  of  the 
simulation  but  the  information-processing  demands  it  imposes  on  the  learner.  He  has 
referred  to  this  characteristic  of  a  simulation  as  its  "cognitive  realism,"  the  degree  to  which 
the  simulation  engages  participants  in  a  decision-making  or  problem-solving  process  that 
parallels  the  mental  activities  required  in  the  real  situation. 

Contrary  to  what  intuitively  may  seem  the  case,  research  does  not  support  the  idea  that 
maximizing  realism  or  fidelity  of  a  simulation  results  in  maximizing  learning  outcomes 
(Ales si,  1987).  With  this  in  mind,  Reigeluth  and  Schwartz  (1989)  have  recommended  that 
the  best  way  to  handle  complexity  in  a  simulation,  when  designing  for  a  novice  learner,  is  to 
start  with  low  fidelity  and  to  add  fidelity  and  complexity  progressively.  Similarly, 
Blumenfeld  et  al.  (1991)  have  proposed  that  a  great  strength  of  simulation  for  instructional 
purposes  is  its  potential  to  allow  students  active  exploration  in  simplified  environments. 
They  believe  that  when  extraneous  details  are  minimized,  interactions  between  variables 
are  easier  to  notice  than  in  a  highly  realistic  simulation  or  in  the  transfer  environment  itself. 

Reigeluth  and  Schwartz  (1989)  have  described  three  major  elements  in  the  design  of  a 
simulation  that  they  believe  determine  its  effectiveness:  the  scenario,  the  underlying  model, 
and  the  instructional  overlay.  They  have  suggested  that  the  scenario  (the  situation  and  the 
learner  interface  with  the  simulation)  and  the  model  (usually  a  mathematical  formula  in 
computer-based  simulations  for  establishing  causal  relationships  but  can  be  some  other 
basis)  should  duplicate  to  some  degree  the  essential  characteristics  of  the  transfer  situation. 
In  other  words,  the  characteristics  of  the  scenario  and  the  model  determine  the  fidelity  of  the 
simulation,  although  how  to  identify  the  essential  characteristics  of  the  transfer  situation  is 
not  addressed.  Reigeluth  and  Schwartz  have  concluded,  on  the  basis  of  their  own  analysis 
of  simulations,  that  the  instructional  overlay  (the  features  in  the  simulation  that  function  to 
optimize  learning  and  motivation)  are  generally  the  weakest  aspect  in  educational 
simulations. 

One  element  of  the  instructional  overlay  that  Reigeluth  and  Schwartz  (1989)  feel  should 
receive  more  attention  from  designers  is  the  provision  of  artificial  feedback.  Alessi  and 
Trollip  (1985)  have  distinguished  between  natural  feedback  that  the  real-life  situation 
provides,  and  artificial  feedback  that  the  designer  builds  into  the  simulation.  One  of  the 
strengths  of  simulation  for  instructional  purposes  is  its  potential  to  shelter  learners  from 
costly  forms  of  natural  feedback  (skidding  into  a  snow  bank)  and  to  provide  real-time 
artificial  feedback  (turn  in  the  direction  of  the  skid.) 

The  simplifying  conditions  method  proposed  by  Reigeluth  (1993)  appears  to  take 
advantage  of  strengths  inherent  in  simulation  without  sacrificing  authenticity  of  the  learning 
activity.  In  this  method,  experts  identify  a  simple  kind  of  case  that  is  as  representative  as 
possible  of  a  real-world  iask  and  the  ways  in  which  this  "epitome"  version  ot  the  task  differs 
from  more  complex  versions.  Over  time,  complexity  and  variation  are  added  to  the  learning 
activity  in  a  systematic  manner  with  the  expectation  that  the  method  preserves  the 
potential  benefits  of  in-context  learning.  Reigeluth  has  claimed  that  this  is  a  more  holistic 
way  to  sequence  instruction  than  the  traditional  parts-to-hole  approach  and  is  compatible 
with  context-based  design  models.  In  the  following  section,  we  will  return  to  this  and  other 
issues  previously  raised  when  we  discuss  what  we  see  as  the  similarities  and  differences 
between  real-life  situations,  simulations,  and  authentic  learning  activity. 

Question  3.  What  are  the  characteristics  of  an  authentic  learning  activity,  and  how  is  it 
related  to  problem-solving  in  real  life  and  simulated  situations? 
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Buchanan  (1992)  has  suggested  that  since  conditions  are  ill-structured  and  problems 
are  ill-formulated  in  many  areas  of  human  endeavor,  all  but  the  most  clearly  linear  design 
problems  assume  a  fundamental  indeterminacy,  von  Bertalanffy's  (1967)  distinction 
between  open  as  opposed  to  closed  systems  is  relevant  to  this  view.  He  has  written  that 
closed  systems  such  as  cybernetic  or  feedback  systems  are  open  to  information  but  do  not 
exchange  matter  with  the  environment.  Open  systems,  on  the  other  hand,  such  as 
organisms  and  other  living  systems  are  maintained  in  a  continuous  exchange  of  components. 
Instructional  design  within  an  open  systems  framework  requires  a  shift  in  preferred 
metaphor  for  education  from  transmission  of  information  to  building  representations  of 
meaning. 

From  a  constructivist  perspective,  the  most  pertinent  issue  facing  designers  is  what 
educational  goals  are  worthwhile.  Authentic  activity  represents  a  holistic  and  generative 
view  of  appropriate  learning  activity  that  treats  learning  and  motivation  as  interdependent 
processes  and  places  emphasis  on  self-directed  learning  and  on  development  of 
metacognitive  ability  necessary  to  support  it.  Authentic  learning  situations  retain  some  of 
the  complexity  and  messiness  of  real-world  problem-solving  situations,  as  well  as  some  of 
the  advantages  of  simulation  discussed  previously. 

One  way  to  think  about  authentic  learning  activity  is  as  a  simulation  where  the 
instructional  overlay  is  designed  to  support  a  related  set  of  values:  collaboration,  autonomy, 
multiple  perspectives,  pluralism,  activity,  reflectivity,  generativity,  authenticity,  and 
ownership  (Lebow,  1993).  In  this  view  of  instruction,  ends  are  integrated  with  means.  For 
example,  a  goal  of  instruction,  to  develop  interpersonal  skills  for  sustaining  cooperative 
group  work,  is  also  a  means  to  achieving  the  very  same  goal  by  practicing  group  process 
skills  in  the  context  of  personally  relevant  goals.  Another  goal,  to  develop  the  ability  to 
reflect  on  one's  own  learning  processes,  is  also  a  means  to  self-correction  and  self- regulation 
of  the  learning  process.  In  effect,  instruction  within  an  authentic  learning  activity  is  a  model 
for  the  values  that  instruction  is  designed  to  support. 

Carroll  (1990)  has  suggested  that  in  order  to  facilitate  transfer,  promote  metacognitive 
and  affective  learning,  support  an  adaptive  motivational  pattern  to  learning  and  encourage 
a  high  degree  of  ownership  and  personal  relevance,  educators  should  provide  training  on  real 
tasks.  Similarly,  Spiro,  Vispoel,  Schmitz,  Samarapungavan,  and  Boeger  (1987)  believe  that 
"cases  and  examples  must  be  studied  as  they  really  occur,  in  their  natural  contexts,  not  as 
stripped  down  'textbook  examples'  that  conveniently  illustrate  some  principle"  (p.  181). 
Honebein  et  al.  (1994)  have  concluded  that  understanding  developed  in  a  simplified 
environment  is  different  from  understanding  in  the  transfer  environment.  They  have  argued 
that  the  complexity  of  the  learning  environment  in  the  early  stages  of  learning  should  reflect 
the  complexity  of  the  authentic  context  to  the  extent  practical.  Otherwise,  when 
instructional  designers  simplify  the  learning  environment,  they  may  unwittingly  alter  the 
metacognitive  and  affective  demands  of  the  authentic  task  complex.  From  this  perspective, 
the  role  of  instruction  changes  from  controlling  student  learning  through  imposing  a 
simplifying  structure  on  the  environment  to  developing  n*w  strategies,  tools,  and  resources 
that  support  the  student  in  functioning  within  the  authentic  learning  context. 

Our  understanding  of  authentic  activity  as  a  model  for  appropriate  learning  activity  is 
somewhat  different  than  the  one  expressed  above.  W r.  see  authentic  activity  as  involving  a 
more  complex  view  of  fidelity  than  simply  a  concern  for  optimizing  the  degree  of  realism  and 
the  level  of  complexity  in  a  learning  situation.  Anderson  (1990)  has  suggested  that  if  we 
want  to  explain  human  behavior,  we  should  seek  understanding  in  the  individual's 
assumptions  about  the  environment  and  in  the  information-processing  demands  that  the 
environment  imposes.  On  this  basis,  the  optimal  degree  of  fidelity  and  complexity  required 
in  a  simulation  to  effectively  transfer  new  learning  is  determined  by  the  affordances  of  the 
environment  and  the  frame  of  reference  of  the  learner.  What  matters  most  is  not  the 
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realism  of  the  simulation  or  the  processing  level  of  the  learner,  but  whether  the  learner 
practices  what  is  essential  for  the  transfer  situation.  Thus,  a  good  test  is  also  a  good 
learning  activity  and  instruction  and  assessment  are  inseparable  within  an  authentic 
learning  (  nvironment  (Snow  &  Mandinach,  1991). 

A  fundamental  principle  of  instructional  design  that  follows  from  an  open  systems  view  is 
that  the  c  Mentation  of  the  individual  to  learning  is  part  of  the  context.  For  example, 
students  ften  study  so  as  to  produce  the  outcome  that  they  expect  the  teacher  to  assess 
(Schmeck*  1988).  When  authentic  activity  is  the  model  for  learning  activity,  the  meaning 
that  the  individual  attributes  to  learning,  including  expectations,  attitudes,  and  beliefs,  is  a 
focus  for  the  design  effort.  From  a  post-modern  perspective,  the  individual's  perceptions 
have  value  and  represent  a  basis  for  mutual  inquiry  rather  than  an  obstacle  to  be 
maneuvered  around  (Doll,  1989;  Gough,  1989). 

In  summary,  when  authentic  activity  is  the  model  for  appropriate  learning  activity,  the 
perceptions  of  the  learner  and  the  affordances  of  the  environment  represent  an  integral  and 
inseparable  context  of  learner/environment.  The  implications  for  instruction  are  primarily 
twofold:  design  must  support  the  learner  in  establishing  a  learning  enterprise  within  the 
larger  global  task  environment,  and  the  learning  situation  must  afford  the  kinds  of  activities 
that  are  essential  for  success  in  the  transfer  environment. 
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I.  Introduction 


American  public  education  has  periodically  experienced  the  need  for  restructuring.  In  recent 
decades,  there  has  been  increasing  a  perception,  research  and  practice  in  fundamental  restructuring. 
A  systemic  approach  to  school  restructuring,  Educational  Systems  Design,  has  emerged  in  response 
to  questions  regarding  the  main  trend  of  current  education  and  its  efforts  toward  change  . 
Researchers  have  introduced  the  principles  and  processes  of  educational  system  design  by  reflecting 
the  relationship  between  educational  systems  and  their  environment  (Banathy,  1991;  Ba  athy,  1992; 
Reigeluth,  1991;  Reigeluth,  1993;  Lieberman,  Zuckerman,  Wilkie,  Barinas,  &  Hergert,  1991).  These 
writers  share  the  underlying  belief  that  changes  in  society  generate  the  need  for  concurrent  changes 
in  education. 

With  this  in  mind,  it  is  of  the  greatest  importance  that  stakeholders  who  participate  in 
redesigning  their  educational  system  examine  and  clarify  their  individual  values  concerning 
learners,  and  education  and  its  relationship  to  the  environment,  and  that  they  articulate  shared 
values,  and  state  them  clearly.  This  is  important  because  values  underlie  visions,  guide  choices, 
decisions,  and  actions  made  in  the  course  of  design,  and  serve  as  a  practical  tool  for  evaluation 
(Rokeach,  1973;  Isaacson  &  Bamburg,  1992;  Checkland,  1981;  Banathy,  1991;  Mohrman  & 
Cummings,  1989;  Senge,  1990;  Pruzan  &  Thyssen,  1994;  Goulet  and  Dolbec,  1991;  Fambrough, 
1991). 

Recent  theoretical  research  (Banathy,  1991;  Banathy,  1992;  Reigeluth,  1993;  Meyer  &  Pruzan, 
1991;  Sergiovanni,  1989;  Schlecthy  &  Cole,  1992;  Miles  &  Ekholm,  1991;  Smith  &  O'Day,  1990; 
Rehni.  Schweitz,  &  Granata,  1992;  Nadler,  1981;  Ackoff,  1981)  addresses  the  need  for  articulating 
values  in  all  stakeholder  groups  and/or  further  suggests  frameworks  for  exploring  options  for 
educational  system  design.  Empirical  studies  support  the  notion  that  values  explicated  und  shared 
by  stakeholders  are  one  of  the  key  elements  in  the  success  of  educational  system  design 
(Breidenbach,  1989;  Lieberman,  Zuckerman,  Wilkie,  Barinas,  &  Hergert,  1991;  Reigeluth,  Norris.  & 
Ryan,  1991;  Reece,  1991).  The  findings  of  these  studies,  however,  remain  at  a  superficial  or  general 
level. 


II.  Conceptual  Framework  and  Literature  Review 

Educational  System  Design  and  Values 

Banathy  (1991)  addresses  values  clarification  as  a  core  process  following  re-visioning  of  the 
future  educational  system  and  preceding  to  create  a  new  image  of  educational  systems  that  is 
compatible  with  the  larger  society.  He  notices,  however,  that  those  processes  are  fundamentally 
iterative  due  to  the  "feedforward  and  feedback  nature  of  the  inquiry"  (Banathy,  1991).  Reigeluth 
(1993)  echoes  Banathy's  notion  by  asserting  that  consensual  values  among  stakeholders  should  be  a 
must  for  them  to  agree  on  since  different  values  by  their  nature  will  lead  to  differe  it  images  and 
features  of  the  system. 

The  ethical  processes  related  to  systems  design  (Meyer  &  Pruzan,  1991;  Pruzan  &  Thyssen, 
1994)  have  been  an  increasingly  debated  issue.  In  an  ethical  approach,  "a  broad  cross-section  of 
stakeholders"  (Weisbord,  1992,  p.  5)  participate  in  values  clarification  through  ongoing  value-based 
dialogue.  Thus,  communication  regarding  stakeholders'  values  is  considered  to  be  a  necessary 
condition  as  well  as  a  result  of  the  self-designing  capabilities  of  an  educational  system  (Meyer  & 
Pruznn,  1991). 

Reigeluth  proposes  to  examine  societal  and  learner  needs  by  carefully  analyzing  current 
change  and  hypothesizing  about  future  change  in  our  society  (Reigeluth,  1993,).  Banathy  (1991; 
\\)\Y2)  proposes  a  more  multi-dimensional  and  comprehensive  framework  designed  to  help 
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stakeholders  self-reflect  on  their  current  system,  articulate  core  values  and  ideas,  and  synthesize 
these  values  and  ideas  into  a  new  educational  system. 

Thus  Banathy's  framework  demands  that  stakeholders  make  decisions  relevant  to  choices 
among  a  great  number  of  alternatives.  As  they  encounter  these  decision-making  points,  dialogue 
among  the  stakeholders  evolves  around  the  question  of 9 why9  a  given  decision  should  be  as  its  is  and 
not  other  wise.  The  conversational  process  for  answering  this  question  leads  stakeholders  to  realize, 
articulate,  and  consent  to  values;  as  Banathy  (1992,  p.  34)  states: 

"As  they  probe  into  the  WHY's  of  a  decision,  they  begin  to  articulate  their  values, 
assumptions,  and  knowledge/experience  base  that  underlie  and  support  a  particular  decision. 
These  emerging  'rationales  for  making  decisions'  will  produce  an  ever-increasing  wealth  of 
descriptions  of  values  and  ideas.  Those  which  they  agree  upon  become  the  'CORE  VALUES' 
and  'CORE  IDEAS'  that  will  continuously  guide  -  even  drive  -  the  design." 

Furthermore,  Banathy  (1992)  points  out  the  importance  of  recognizing  two  essential  qualities 
of  values  in  systems  design.  First,  one  should  consider  the  emergence  of  values  as  an  evolutionary 
process.  As  one  proceeds  with  systems  design,  substantial  core  value  are  reinforced,  modified,  and 
realized.  Second,  one  should  pay  attention  to  internal  consistency  among  values  since  all  dimensions 
are  interactive  and  interdependent.  Hence,  each  decision  and  the  values  behind  it  have  to  be 
approached  within  the  systemic  context  of  the  framework  by  seeking  and  re-ensuring  internal 
consistency. 

Schlechty  and  Cole  (1992)  propose  that  systemic  standards  should  be  created,  and  their  basis 
should  be  a  set  of  values  concerning  how  schools  ought  to  be.  Then,  this  coherent  set  of  values  and 
beliefs  can  provide  the  ground  work  for  envisioning  a  system.  The  Center  for  Leadership  in  School 
Reform  (Schlechty  and  Cole,  1992)  began  with  10  values  when  it  worked  with  schools,  school 
districts,  and  communities  for  systemic  change.  This  values  list  presented  multiple  levels  of  values 
by  reflecting  learners*  characteristics  and  those  of  the  organizational  and  societal  levels.  As  the 
authors  mention  in  their  article,  however,  these  values  were  imposed  on  local  stakeholders  as  a  norm 
(although  local  stakeholders  agreed  to  use  them).  Based  on  previous  research  (Banathy,1991; 
Reigeluth,1993;  Meyers  &  Pruzan,  1991;  Pruzan  &  Thyseen,  1994),  values  articulated  in  this  way 
without  stakeholders1  participation  cannot  function  as  well  as  it  should.  Moreover,Schlechty  and 
Cole  do  not  provide  the  readers  with  the  values  criteria  used  to  examine  the  comprehensiveness  of 
those  values  that  are  used  to  be  creating  systemic  standards. 

On  the  other  hand  ,  Banathy  (1991)  presents  three  categories  of  values  as  well  as  examples  of 
values  in  each  category.  While  emphasizing  that  stakeholders  should  develop  their  own  sets  of 
values,  sets  of  terminal  and  instrumental  values  are  presented  based  on  Banathy's  perspective  on 
the  categories  of  educational  functions  and  purposes,  learner  and  learning,  and  systems  design  that 
will  guide  chcnge. 

There  are  several  case  studies  that  examine  educational  systems  design  efforts  and  their 
outcomes.  These  attempt  to  identify  the  main  factors  contributing  to  or  hindering  the  success  of 
change  and  to  draw  conclusions  or  make  recommendations  that  are  generalizable  to  any  public 
educational  sysvem,  based  on  n  systemic  view. 

Breidenbach  (1989),  in  her  case  study  of  u  major  metropolitan  public  school  district,  found  that 
the  values  and  beliefs  shared  by  advocates  were1  one  of  the  10  main  factors  contributing  to  this 
district's  successful  design  process.  In  a  nationwide  rune  study,  Reigcluth,  Norris,  and  Ryan  (1991) 
also  found  values  (beliefs)  to  be  one  of  the  keys  to  rt  Huvrensful  change  process.  Meanwhile, 
Hewed  991)  emphasizes  values,  especially  bused  on  local  heeds,  as  one  of  the  6  major  components  in 
his  case  study  examining  the  restructuring  practices  implemented  by  a  school. 

From  the  above  literature  review,  the  follow  in"  conclusions  can  he  drawn.  Articulation  of 
values  should  be  a  front-end  step  in  educational  sy  r  ins  design  and  should  be  formulated  with 
respect  to  the  interrelationship  of  systems  and  the.  environment.  However,  none  of  the  available 
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studies  provide  stakeholders  with  substantial  guidance  for  articulating  their  core  values.  Given  the 
importance  of  values  clarification  by  stakeholders  and  the  lack  of  previous  in-depth  studies,  it  is 
important  to  establish  comprehensive  categories  to  aid  in  the  articulation  of  core  values  and  to  serve 
as  guidelines  for  applying  these  values  in  the  design  of  educational  systems. 

III.  Objectives  and  Research  Questions 

The  overall  purpose  of  this  research  is  to  determine  comprehensive  categories  of  values,  as  well 
as  the  values  in  each  category  that  need  to  be  articulated  and  consented  to  by  stakeholders  through  a 
qualitative  case  study  approach.  The  term  values,  as  used  in  this  study,  is  defined  as  the  ideals  that 
a  design  tries  to  approximate  (Ackoff,  1981,  p.  125,  calls  it  '"ultimate  values")  regardless  of  means  or 
ends,  or  either  values  or  ideas  (Banathy  distinguished  between  these  two  concepts). 


IV.  Methodology 

The  present  research  approach  involves  a  qualitative  case  study  methodology.  As  Cronbach 
(1975)  points  out,  a  qualitative  case  study  allows  for  "interpretation  in  context  (p.  123)."  Especially 
in  addressing  'why1  questions,  the  fundamental  questions  necessary  to  clarify  values,  case  study  has 
been  an  effective  strategy  (Yin,  1986).  In  addition,  a  case  study  approach  can  help  a  researcher 
explain  the  background  of  a  situation  as  well  as  what  happened,  include  vivid  materials  such  as 
interviews,  quotations,  newspaper  articles,  and  the  like,  obtain  information  from  a  wide  variety  of 
sources,  present  it  in  a  v- -«  ^ty  of  ways,  and  suggest  to  readers  what  to  do  or  not  do  in  similar 
situations  (Hoaglin,  Light,  McPeek,  Mosteller,  and  Stoto,  1982). 

Research  site 

Using  a  purposive  sampling  strategy  (Kidder,  Judd,  and  Smith,  1984;  Merriam,  1988),  an 
elementary  school  was  selected  for  analysis  in  the  study  based  on  the  following  criteria: 

•  A  public  school  that  is  currently  undergoing  systemic  restructuring 

•  A  school  that  has  values  articulated  and  shared  by  stakeholders 

•  A  school  that  is  in  the  relatively  early  implementation  stage  so  that  participants  still  have 
vivid  memories  of  which  values  they  have  identified  or  ignored  and  of  why  and  how  they 
have  done  so.  In  addition,  the  impact  of  planning  can  be  more  easily  distinguished  in  the 
early  stages  of  implementation  than  later,  after  many  new  factors  have  emerged. 

•  For  convenience,  however,  I  arbitrarily  limited  the  study  to  a  school  in  the  area  of  the 
researcher's  residence. 

Contact  and  consent 

An  initial  contact  with  this  school  was  made  through  a  phone  call  to  the  principal.  Before 
starting  each  research  interview,  I  explained  the  purposes  and  scope  of  the  study  and  formally 
obtained  a  signed  consent  form  from  each  informant. 

Data  Collection  Techniques  and  Procedures 
Prior  to  the  first  site-visit,  1  analyzed  available  documents,  including  an  annual  report,  local 
newspaper  clips,  and  some  promotional  pamphlets  which  were  already  available.  This  provided  me 
with  preparatory  information  about  the  site.  Upon  visiting  the  school,  I  discussed  my  research  plan 
with  the  principal.  At  this  pre-research  meeting,  I  obtained  more  documents,  which  included 
Indiana  2000  application  and  PBA(Performance  Based  Accreditation)  documents.  These  materials 
allowed  nie  to  gain  an  understanding  of  the  general  context  of  the  school,  which  included  historical, 
demographic,  and  social  background  relevant  to  the  restructuring  process. 
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,  The  primary  data  collection  technique  involved  interviews  with  the  stakeholders  most  involved 
in  redesigning  of  their  school  system.  These  induded  an  administrator,  two  teachers,  two  parents, 
and  one  community  member.  I  mainly  conducted  semi-structured  interviews,  since  my  research  was 
focused  on  approaching  the  perspectives  of  the  informants  being  interviewed  (Patton,  1980).  These 
interviews  were  guided  by  a  list  of  questions  or  issues  to  be  explored  (See  Appendix  A:  Interview 
Questionnaires),  but  neither  the  exact  wording  ^or  the  original  order  of  the  questions  was  used 
during  the  interviews.  In  addition,  structured  .;:terviews  were  added  to  obtain  some  standardized 
information  ( Merriam,  1988).  Information  was  collected  about  the  past,  the  present  and  the  future 
by  asking:  What  has  the  school  done,  how,  and  why?  What  have  they  been  doing,  how,  and  why? 
What  are  they  doing,  how,  and  why? 

Key  informants  were  interviewed  individually  at  least  twice.  Follow-up  interviews  were  made 
either  through  a  meeting  or  phone  call.  I  took  notes  during  the  interviews,  which  were  audio  tape* 
recorded.  Verbatim  transcriptions  were  made  from  the  recorded  interviews.  This  practice  ensured 
that  everything  said  was  preserved  for  analysis  (Merriam,  1988).  As  a  "non- participant  observer" 
(Merriam,  1988)  I  observed  classroom  activities.,  after  I  left  the  room  and  summarized  my 
observations  as  soon  as  possible  either  on-  or  off-site.  Then  I  took  time  to  remember  more  substance 
and  elaborated  the  scenes  and  dynamics  at  the  site.  Table  1:  Calendar  of  Data  Collection  depicts 
the  order  of  the  interviews,  document  collection,  and  observations  as  they  occurred.  Data  from 
documents  and  observations  were  recorded  for  the  purpose  of  verification  purpose. 

Data  Analysis  Techniques 
The  techniques  of  Unitizing'  and  'categorizing'  (Lincoln  &  Guba,  1985,  p.  344)  were  used  to 
analyze  data.  Data  analysis  frequently  began  with  a  review  of  my  research  proposal,  as  Goetz  and 
LeCompte  (1984)  suggest.  Then,  all  data  from  interview  transcripts,  documents,  and  observation 
notes  were  read  through  several  times  from  beginning  to  end.  While  reading,  I  jotted  down  notes, 
comments,  and  observations  in  the  margins  (Goetz  and  LeCompte,  1984).  Through  this  process,  I 
identified  units  of  information  as  the  basis  for  defining  categories.  Then,  selected  information  was 
summarized  and  categorized  through  the  use  of  codes.  (See  Appendix  B:  The  Summary  of  Contact 
Form). 

A  ' start  list  of  codes'  (Appendix  C:  Start  List  of  Codes)  was  created  in  the  early  stages  of  data 
collection.  This  list  was  inductively  developed  based  on  findings  from  the  document  analysis  and 
some  initial  interviews.  Definitions  of  codes  (Appendix  D:  Definition  of  Codes)  were  developed  for 
each  code  since  defining  codes  helped  the  researcher  to  consistently  apply  codes  over  time.  The  codes 
in  this  list,  however,  changed  and  developed  as  research  experience  continued.  Some  codes  did  not 
work  and  t  were  thus  discarded  or  reorganized.  Other  codes  became  too  general,  resulting  in 
subcodes.  However,  new  codes  still  emerged  progressively  during  data  collection.  Accordingly, 
coding  was  a  form  of  continuing  analysis  (Merriam,  1988)  which  allowed  for  inductive  examination 
of  data. 

Interview  data  analysis 

The  data  consisted  of  verbal  accounts,  coming  from  3-4  hours  of  interview  interactions. 
Interview**  were  conducted  between  late-March  and  Mid-June,  1993.  Upo~  completion  of  all 
interviews,  the1  contact  summaries  were  edited  and  sent  to  the  informants.  Each  informant  was 
culled  alter  they  received  the  summarized  interviews  -interview  summary  reports  -for  initial 
verification  of  content.  All  informants  but  the  community  member  verified  the  interview  summary 
report  h  (See  Appendix  K:  Interview  Summary'  Report).  Minor  editing  was  done  through  phone  calls 
or  personal  contacts  according  to  participants'  preferences.  Upon  conducting  these  verification 
cnnturtH,  there  wan  one  follow-up  interview. 
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February  18, 1993 

Exploratory  phase  of  research 

March  4, 1993 

First  meeting  with  principal;  collection  of  several  documents;  short 
tour  of  building 

March  11, 1993 

Final  permission  obtained  from  principal;  received  potential 
interviewees'  list 

March  26, 1993 

First  research  interview  with  principal 

April  2, 1993 

Second  interview  with  principal 

April  20,1003 

First  interview  with  VR;  collected  an  additional  document 

April  29, 1993 

First  interview  with  MM;  briefly  visited  computer  lab 

May  6,  1993 

First  interview  with  CT;  second  interview  with  MM;  collected 
documents 

May  7,  1993 

First  interview  with  KW 

May  11,1993 

Observed  VR  s  and  CD  s  morning  classes 

Mary  12,  1993 

Observed  morning  classes  at  the  computer  lab 

May  13, 1993 

Second  meeting  with  CT 

Mary  14, 1993 

Second  interview  with  KW 

May  20, 1993 

First  interview  with  MMc 

May  25,1993 

Second  inten/iew  with  MMc 

June  1-4,  1993 

Delivered  interview  summary  reports  for  member  checks 

June  10-18, 1993 

Follow-up  interview  to  validate  and  revise  summary  reports 

Aug.  16,  1993 

Data  collection  for  research  question  #2  (primary  value  categories) 
based  on  the  findings  done  through  tho  primary  analysis 

Table  1.  Calendar  of  Data  Collection 


Document  Analysis 

Documents  were  also  summarized  and  ended  on  tin1  summary  hIh'H.  Thorn1  Hiimmiirv  forma 
were  used  for  analysis  but  also  facilitated  rapid  retrieval  whi»n  iic»ri»HHnry  I'rlmurily,  tin1  content 
analysis  focused  on  measuring  the  frequency  and  variety  of  infiirtnnt iuti,  cunflruiitiK  the  omet^nt 
hypotheses,  and  advancing  new  categories  and  hypotluwH  (Merrium,  1HHH). 
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Observation  analysis 

Data  collected  through  observation  was  summarized  and  coded  on  observation  summary 
sheets.  The  main  focus  in  the  observation  was  to  answer  the  question,  "Are  they  really  practicing 
what  they  said  they  value?"  The  main  purpose  of  the  observation  was  for  triangulation. 
Observation  through  the  analysis  of  facts  in  detail  was  attempted  to  (1)  identify  the  practices  that 
were  relevant  to  the  value  categories  identified  through  interviews  and  document  analysis  and  (2) 
examine  any  significant  inconsistencies  between  values  and  practice. 

Developing  categories  involves  seeking  recurring  regularities  in  data  by  comparing  one  unit 
of  information  with  the  next.  Once  the  relevant  sets  of  categories  are  derived  from  the  data,  they  can 
be  filled  out  and  made  more  vigorous  by  searching  through  the  data  for  more  and  better  units  of 
information  (Merriam,  1988,).  Through  these  processes,  the  sets  of  value  categories  were  formulated 
to  reflect  the  interrelationships  of  the  value  components.  An  alternative  display,  a  matrix,  was 
utilized  for  better  analysis  (See  Appendix  F:  Matrix) 

Upon  completion  of  data  analysis,  the  emergent  value  categories  were  sent  to  informants  in 
order  to  find  the  data  for  research  question  2,  "primary  and  secondary  value  categories*"  Thu 
list  of  value  categories  was  provided  to  participants  with  Likert  Scaling,  which  has  a  code  from  1  to 
5,  from  least  to  most  important.  ( See  Appendix  G:  Primary  vs.  Secondary  Value  Categories 
Inventory.)  Definitions  of  value  categories  were  provided  on  a  separate  sheet  in  order  to  help  the 
informants  have  a  clearer  idea  about  each  term  in  the  categories  (See  Appendix  H:  Definitions  of 
Value  Categories.)  Then,  I  computed  the  total  scale  scores  for  each  value  category  by  summing 
informants'  scores  on  all  the  values.  This  process  was  un  additional  attempt  to  construct 
"negotiated  outcomes"  Lincoln  &  Guba,  1985,  p.  41)  which  is  done  through  negotiation  of 
meanings  and  interpretations  with  the  informants  from  which  the  data  has  been  chiefly  obtained. 

Cmaimt  pi  Study  Sites 

Demographics  of  school  building 

The  study  site  was  a  relatively  large  elementary  school  of  530  students.  The  faculty  describe 
their  school  as  being  somewhere  between  rural  and  suburban.  However,  they  think  they  are  close  to 
being  a  suburban  school. 

There  were  drastic  changes  and  growth  in  the  school  community  between  1989  and  1991  with 
the  construction  of  a  new  large  school  building  in  1990.  50%  of  the  families  and  75%  of  the  faculty 
are  new.  In  1989,  the  school  had  1 1  staff  members  but  by  1992  it  had  52.  Most  parents  were 
employed  in  light  manufacturing  or  service  sector  jobs.  Four  to  five  percent  of  the  students  are 
ethnic  minorities,  and  some  use  English  as  a  second  language.  One  of  the  faculty  .members  said  that 
most  of  the  families  are  non-conventional,  that  is  two-parent  and  two-income,  or  single  parent  and 
single  income  families.  10%  of  the  school  population  is  enrolled  in  the  before  and  after  school 
custodial  program. 

Process  of  restructuring 

The  current  restructuring  effort  was  initiated  by  the  faculty  who  saw  that  what  they  were  doing  was  not 
successful  and  thus  wanted  to  make  changes.  (See  Table  2:  Calendar  of  Events  for  their  restructuring 

history.) 
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Table  2. 

Calendar  of  Events 


V.  Final  Analysis 

Categories  of  Vahips 

The  definitions  for  the  value  categories  came  from  specific  data  sources  in  the  research.  1 
searched  informants'  values,  identified  them,  and  labeled  them.  Once  labeled,  I  defined  them 
through  my  own  interpretations.  For  example,  nature  of  a  learner  describes  the  fundamental 
attributes  of  a  learner,  the  ways  a  student  learns  better,  and  the  ways  he  or  she  demonstrates  what 
he  or  she  learns. 
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Category  1:  Nature  of  a  Learner  and  Learning 

There  emerge  two  value  categories  from  the  informants1  dialogues:  the  value  area  in  regard  to 
the  essential  natures  of  a  learner  as  well  as  learning. 

Nature  of  a  learner.  In  this  category,  informants  fleshed  out  their  values  and  ideas  about 
what  fundamental  attributes  a  learner  possesses.  They  talked  about  individual  learners'  attributes 
in  a  variety  of  aspects.  They  also  discussed  about  how  a  learner  learns  better. 

Nature  of  learning.  Informants'  assertions  in  this  category  can  be  grouped  into  two  subject 
areas.  First,  what  are  the  attributes  of  learning?  Second,  what  should  'learning'  do  for  learners? 

Category  2:  General  Principles  of  Restructuring 

The  category  of  general  principles  of  restructuring  appears  to  be  a  significant  value  area 
that  was  frequently  discussed  among  the  informants.  This  category  describes  fundamental  and 
underlying  values  of  school  restructuring.  This  category  emanates  from  the  following  values 
identified  from  different  informant  sources: 

The  reasons  for  restructuring.  Informants  appeared  to  discuss  the  reasons  for 
restructuring  in  terms  of  gaps,  which  might  occur  due  to  changes  in  the  larger  society  and  student 
population,  or  between  needs  and  educational  reality. 

The  attributes  of  restructuring.  Informants  discussed  the  fundamental  qualities  that  make 
restructuring  "a  real  restructuring,"  not  something  else.  They  talked  about  these  attributes  in 
regard  not  only  to  the  process  of  change  but  also  to  the  ultimate  target  for  change. 

The  ultimate  goals  of  restructuring.  Respondents  expressed  their  values  about  for  whom 
or  for  what  reason  they  should  restructure,  that  is,  the  ultimate  beneficiaries  of  restructuring. 

Category  3:  Procedural  Principles 

Procedural  principles  for  school  restructuring  appeared  to  be  another  vital  value 
category.  This  category  describes  any  strategic  value  or  idea  relevant  to  the  factors  leading  to 
successful  restructuring.  This  procedural  principles  category  became  apparent  from  a  wide  range  of 
values  the  informants  discussed,  and  four  general  themes  were  especially  apparent. 

(1)  People:  The  informants  clarified  their  values  with  regard  stakeholders  who  were  to  be 
involved  in  the  restructuring  process.  The  relevant  issues  include  staff  development,  administrative 
leadership,  parent  involvement,  and  community  involvement. 

(2)  Structure:  The  informants  also  discussed  organizational  structural  issues,  which  included 
communication  and  choice  as  critical  factors  in  the  change  process. 

(3)  Culture:  Respondents  spoke  about  ownership,  collective  wisdom,  macro*perspectives,  and 
caring  as  be  required  during  the  restructuring  process. 

(4)  Resources:  They  commented  on  resources  to  be  attained  for  assuring  or  helping  their 
restructuring  process. 

Category  4;  Outcome  of  Learning 

It  appeared  that  the  category  of  'outcome  of  learning1  was  a  significant  value  area  overtly 
discussed  among  the  informants,  which  is  related  to  what  they  wanted  students  to  know  and 
demonstrate  when  they  leave  school.  Many  of  their  dialogues  in  this  category  reflected  their  values 
about  the  nature  of  learners  and  learning  and  some  of  the  general  principles  of 
restructuring.  Their  articulated  values  about  learning  outcomes,  in  general,  appeared  to 
emphasize  the  following  concepts:  'a  whole  person  -  more  than  just  intellectual  excellence/  'self- 
driven  and  continuous  process/  'challenge  and  achievement/  and  'real  life/ 

Category  5:  Process  of  learning 

A  process  of  learning  value  category  emerged,  which  described  the  value  area  regarding  the 
means  for  achieving  learning,  that  is,  how  students  should  achieve  expected  learning  outcomes  and 
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how  schools  should  help  students  learn.  Most  of  the  values  clarified  by  the  informants  in  this 
category  were  consistent  with  their  beliefs  about  'the  nature  of  learner  and  learning.*  It 
appeared  that  the  following  themes  were  the  most  common  among  the  informants:  learning 
arrangement  and  programs,  learning  strategies  and  methods,  learning  situation,  and  the  role  of 
learners  in  the  learning  process. 


The  category  of  assessment  emerged  from  among  the  frequently  discussed  themes  as  follows. 
The  nature  of  assessment  was  discussed  with  respect  to  the  nature  of  learners,  that  is,  individual 
iliHeronreM   The  purpose  of  assessment  also  was  apparent  in  the  informants1  discussions,  being 
mainly  diseiiHsed  in  terms  of 'progress.'  Informants  also  discussed  the  parties  involved  in 
evaluation,,  Ultimately,  the  informants  seemed  to  value  collaborative  assessment  involving  both 
paieiifH  ami  children. 

I  atcgui  v  7*.  Lmnuiig  Environment 

Hie  category  of  'learning  environment*  emerged  from  the  clarified  values.  This  category 
ileHi  iihcH  I  he  hcIiooI  environment,  which  is  considered  to  support  the 1 learning  process9,  and  which 
in  I  ill  ii  is  tha)  (Icftigned  to  ensure  that  learners  achieve  expected  'learning  outcomes.9  The 
mini  mailt  h  appeared  to  address  most  frequently  the  following  themes  in  this  value  category.  First, 
they  exprcMH  the  value  regarding  the  nature  of  the  learning  environment.  Second,  the  scope  of 
the  learning  environment  was  defined,  which  tends  to  be  broader  in  that  it  emphasizes  interaction 
with  tin*  ontHhle  community.  Third,  they  spoke  of  resources  as  a  tool  for  promoting  the  achievement 
nl  educational  goals.  Their  comprehensive  perception  of  resources,  including  both  internal  and 
external  compmicntN,  appeared  to  consistently  reflect  their  values  about  the  scope  of  the  learning 
environment. 

Cui^uuiy  ^Urtfuni^utiuiml  Structure  &  Culture 

The  category  of  organizational  structure  and  culture  emerged  from  the  values  discussed 
by  t  he  informants  ami  in  the  documents  analyzed.  These  values  revealed  either  a  formal  or  informal 
organizational  arrangement  (Nadler,  Gerstein,  &  Shaw,  1992).  First,  a  governance  structure  was  an 
apparent  theme  among  the  informants,  who  appeared  to  share  an  identical  value,  that  is,  a  value- 
laden  derision  making  structure  where  all  the  stakeholders  share  responsibility  for  specific  areas. 
This  value  seemed  to  be  well  reflected  in  the  statement,  "Learning  is  a  shared  responsibility  of  the 
child,  staff,  parents  and  the  community'(Annual  Report,  PBA).  Second,  the  roles  and  responsibility 
of  people  at  school  was  frequently  reflected  by  the  informants.  Third,  the  visible  behaviors  and 
attitudes  revealed  in  a  daily  life  nds  discussed  in  a  variety  of  ways(Nadler,  Gerstein,  &  Shaw,  1992).. 

Category  9  Function  of  education  within  the  larger  society 

Another  emergent  value  category  was  function  of  education  within  a  larger  society.  This 
category  describes  informants'  perceptions  about  education  on  a  broader  scope,  in  the  context  of  the 
larger  society.  This  category  appeared  to  be  discussed  in  terms  of  two  themes.  First,  the  informants 
talked  ahout  their  perceptions  relevant  to  the  function  of  education.  Second,  they  expressed  their 
values  ahout  education's  fundamental  relationship  with  other  systems  in  the  larger  society. 


In  order  to  identify  which  value  categories  the  informants  considered  more  significant  in  terms  of 
redesigning  their  educational  system,  once  having  constructed  the  above  value  categories  through 
the  final  analysis  process,  I  provided  each  informant  with  a  list  of  value  categories  along  with  a  5- 
point  scale,  where  a  higher  number  indicates  greater  importanc?e  (See  Appendix  G:  Primary  vs. 


Primary  vs.  Secondary  Value  Categories 


Secondary  Value  Categories  Inventory),  The  study's  definitions  of  value  categories  were  included 
(See  Appendix  D:  Definitions  of  Value  Categories).  This  was  intended  to  help  the  informants  have 
clearer  idea  about  each  term  presented  on  the  inventory  sheet. 
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Figure  1.  Significant*-  Lewd  ofEaA  Value  Category 


Based  on  the  data  presented  above,  Figure  1, 1  found  the  order  of  respondents'  value  priorities 
to  be  the  following: 

1.  Outcomes  of  learning  &  Process  of  learning 

2.  Nature  of  learner  &  Assessment 

3.  Learning  environment 

4.  Function  of  education  within  the  larger  society 

5.  Nature  of  learning 

6.  Organizational  structure 

7.  Organizational  culture  &  General  principles  of  restructuring 

8.  Procedural  principles  of  restructuring. 

In  general,  the  informants  appeared  to  place  a  priority  on  clarifying  values  directly  related  to 
learners  and  their  benefits.  In  other  words,  it  seemed  imperative  for  them  to  clarify  and  get  to  a 
consensus  on  values  regarding  who  learners  are,  what  they  should  learn,  how  they  should  learn,  and 
how  they  should  be  assessed.  On  the  other  hand,  they  showed  less  regard  for  values  related  to  the 
design  process  itself  and  to  school  as  an  organization. 

Interestingly,  the  staff  who  worked  within  the  organization  showed  the  least  concern  about 
organizational  values.  But  a  community  member  attributed  the  highest  numbers  to  this  value 
category.  This  result  confirmed  the  following  comment  by  the  community  member  her  comment, 
"...If  they  think  and  try  to  operate  as  an  organization,  I  think  schools  end  up  with  more  focus.  Staff 
and  students  will  begin  to  have  a  better  understanding  of  what  it  is  they're  trying  to  accomplish  as  a 
part  of  the  organization.  On  the  other  hand  the  school  itself  will  have  a  better  understanding  of 
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their  role  in  the  larger  community  as  an  organization.  And  then  consequently  they  will  become  less 
isolated."  (CT,  4.7)  This  community  person  emphasized  the  macro  level    the  organizational  and 
societal  level    as  did  the  principle  (DF). 

When  I  analyzed  the  responses  of  internal  informants  (staff  and  parents),  'the  nature  of 
learner'  category  appeared  to  be  their  most  primary  concern.  This  can  be  interpreted  to  mean  that 
the  people  working  close  to  the  students  believed  that  consistent  and  shared  understanding  of 
learners'  natures  among  stakeholders  should  be  in  advance  of  any  other  values. 

These  informants  appeared  to  show  he  least  regard  for  'strategic  values/  that  is,  how  to  change 
their  systems  and  programs.  This  finding  may  be  due  to  two  possible  reasons.  One  is  that  the 
stakeholders  may  have  been  unaware  of  the  importance  of  how  to  attain  the  final  goals  of  their  plan. 
Or,  second,  they  may  have  been  more  interested  in  'product*  than  'process/ 

Values  concerning  the  learning  dimension  were  highly  emphasized.  On  the  contrary,  values 
involving  the  organizational  dimension  were  least  emphasize,  which  might  reflect  what  the 
community  member  (CT)  said:  "They  don't  tend  to  think  of  their  school  as  an  organization/'  (CT, 
4.7.) 

Values  regarding  the  social  dimension  were  considered  to  be  relatively  important  (4th  from  the 
top).  This  result  might  reflect  the  fact  that  the  concept  of  educational  systems  being  restructured 
has  been  well  accepted.  This  seems  to  show  that  the  stakeholders  understand  their  school  system  in 
the  context  of  the  larger  society. 

Shared  Values  in  Each  Category 

The  shared  values  were  demonstrated  through  one  further  analysis  of  my  interpretations 
shown  on  the  matrix  developed  primarily  for  identifying  value  categories  (See  Appendix  I:  Matrix). 

Values  regarding  Nature  of  Learners  and  Learning 

1)  Individual  learner  differences  should  be  recognized.  Learners  learn  in  many  different  ways. 
Learners  also  can  show  their  achievement  in  numerous  ways. 

2)  Learners  learn  better  what  they  are  comfortable  with,  within  a  context,  and  when  they  are 
internally  motivated. 

3)  Learning  is  a  process  that  does  not  stop  and  that  also  takes  time. 

4)  Learning  is  making  sense  of  the  world. 

Values  regarding  General  Principles  of  Restructuring 

1)  Education  should  respond  to  change  in  society. 

2)  Restructuring  is  a  process  of  re-examining  and  challenging  the  assumptions  upon  which  we 
practice  and,  when  appropriate,  of  replacing  invalid  assumptions. 

3)  Restructuring  is  a  continuous,  learning  process  involving  a  long-term  plan  and  continuous 
revision. 

4)  Restructuring  is  change  in  people's  mindset  &  attitude  toward  the  nature  of  learners  as  well 
as  change  in  structure. 

5)  Restructuring  should  focus  primarily  on  what  is  best  with  respect  to  each  learner's 
achievement. 

Values  regarding  Procedural  Principles 

1)  Continuous  and  focused  professional  development  and  staff  self-efficacy  are  driving  forces  in 
restructuring. 

2)  Collaborative  and  collegial  administrative  leadership  is  required. 

3)  Supportive  and  involved  parents  are  crucial  for  successful  restructuring  to  occur. 

4)  Diverse  and  strong  ties  with  the  community  are  important. 
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5)  Choice  should  be  always  available  through  'schools  within  a  school:  We  need  to  balance  the 
process  of  supporting  people  who  try  innovation  with  not  loosing  people  who  feel  uncomfortable 
with  change. 

6)  Through  a  value-laden  change  process,  all  stakeholders  should  develop  ownership  from  the 
early  stages  of  change. 

7)  Collective  wisdom  is  better  than  an  individual's  or  small  groups'  ideas. 

8)  Streamlined  communication  is  critical  for  effective  communication  in  the  school. 

9)  Resources,  including  time  to  collaborate,  money,  legitimacy  of  change,  and  networking  with 
other  organizations  will  help  restructuring  happen. 

Values  regarding  Outcomes  of  Learning 

1)  A  whole  person:  A  learner  achieves  excellence  not  only  intellectually  but  also  socially, 
emotionally,  and  physically. 

2)  A  learner  develops  the  ability  to  become  a  productive  and  responsible  life- long  learner. 

3)  A  learner  respects  him-  or  herself,  others,  and  the  environment. 

4)  A  learner  sets  and  meets  higher  standards. 

5)  A  learner  has  problem -solving  skills. 

6)  A  learner  has  high  order  thinking  skills. 

Values  regarding  Process  of  Learning 

An  overarching  core  value  is  that  the  learning  process  should  be  consistent  with  findings  about 
the  nature  of  learners  and  learning.  Based  on  this  belief,  informants  shared  the  following  core  ideas: 

1)  The  process  of  learning  should  allow  for  learning  based  on  individual  uniqueness.  It  should 
allow  for  learning  at  individual  learners'  speeds  and  needs,  be  connected  with  their  learning 
styles,  help  them  feel  comfortable  with  the  learning  process,  and  capitalize  on  learners'  strengths 
and  also  strengthen  their  weaknesses. 

2)  The  process  of  learning  should  allow  for  learning  within  a  context,  that  is  meaning-centered 
learning.  It  should  allow  learners  to  make  sense  of  their  school  experience  by  connecting  it  with 
real  life. 

3)  The  process  of  learning  should  have  a  cumulative  and  consistent  direction  so  as  to  allow  for 
students'  continuous  progress. 

4)  The  process  of  learning  should  promote  learners  taking  more  responsibility  for  their  own 
learning  by  allowing  for  'choice'  by  learners. 

Values  regarding  Assessment 

1)  Assessment  should  be  consistent  with  the  process  of  learning  used  at  CC.  Student  outcomes 
achieved  by  the  learning  program  are  the  things  that  need  to  be  assessed. 

2)  Assessment  should  focus  on  students'  continuous  progress.  Feedback  given  to  learners,  by 
nature,  should  encourage  learners'  continuous  progress. 

3)  Assessment  tools  should  be  comprehensive,  multi- dimensional,  and  non-  artificial. 

4)  Collaborative  assessment  should  be  pursued  in  which  students'  responsibility  should  be 
promoted. 

Values  regarding  Learning  Environment 

1)  Learning  environment  should  'promote  the  richest  opportunity  for  a  child's  success  in 
school'  (PBA):  It  should  be  warm,  supportive,  and  safe. 

2)  Learning  is  not  confined  within  classroom  walls. 

3)  A  variety  of  resources  available  within  both  the  educational  system  and  the  larger  society 
should  be  used  fully  . 
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Values  regarding  Organizational  Structure  &  Culture 

1)  A  value-laden  decision-making  structure  exists.  All  stakeholders  share  responsibility  for 
specific  areas  of  decision  making. 

2)  Parent  involvement  at  the  instructional  level  is  most  important;  students  take 
responsibility  for  and  ownership  of  their  learning;  teachers  coach  with  respect  to  learning; 
administrators  help  and  coach  teachers  by  building  their  strengths  based  on  individual 
differences. 

3)  There  is  a  streamlined  communication  structure  among  staff,  school  and  home,  and  school 
and  community. 

Values  regarding  the  Function  of  Education  within  thq/Lsrg'jr  Society 

1)  Education  should  provide  for  life-long  learning. 

2)  Education  is  a  part  of  the  world.  Education  should  coordinate  with  other  systems  in  the 
larger  society  for  life-long  learning. 

VI.  Conclusions 

This  study  investigated  a  comprehensive  and  systemic  model  of  value  categories  to  be 
discussed  and  elaborated  by  stakeholders  in  redesigning  an  educational  system,  and  the  values  in 
each  value  category  that  have  been  articulated  and  consented  to  by  stakeholders  at  an  elementary 
school,  CC.  Developing  values  and  ideas  shared  among  stakeholders  is  the  fundamental  process  of 
educational  systems  design.  This  is  primarily  because  values  allow  stakeholders  to  elaborate  their 
vision  and  guide  systems  design  (Banathy,  1992).  But  also,  the  process  of  elaborating  and  sharing 
values  is  considered  to  be  the  necessary  condition  as  well  as  a  result  of  the  self-designing  capabilities 
of  an  educational  system(Meyer  and  Pruzan,  1991) 

This  study  identifies  three  value  domains,  comprised  of  10  value  categories  that  emerged  from 
the  main  stakeholders1  values  and  perspectives  (Figure  1.  A  Model  of  Value  Categories). 
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Figure  2.  A  Model  of  Value  Categories 


ERIC 


Smninal  Values 

Seminal  values  are  fundamental  beaefs  that  inspire  the  image  of  a  new  educational  system 
and  lead  the  ^vhole  process  of  restructuring.  These  values  consist  of  two  categories.  The  first  is 
Nature  of  a  learner  and  learning,  reflecting  the  fundamental  attributes  a  learner  possesses  and 
how  a  learner  learns  better,  as  well  as  the  fundamental  attributes  and  functions  of  learning.  These 
value  areas  are  considered  to  be  highly  important  in  terms  of  need  for  clarification. 

The  second  category  is  General  Principles  of  Restructuring,  reflecting  the  reasons  for 
restructuring,  the  nature  of  restructuring,  and  the  ultimate  focus(es)  of  restructuring.  However,  the 
stakeholders  believed  this  value  category  has  no  priority  in  terms  of  their  restructuring  concerns. 

Strategic  Values 

Strategic  values  are  primarily  instrumental  in  helping  restructuring  proceed  or  in  achieving 
the  desired  outcomes  of  restructuring.  These  values  comprise  the  category  of  procedural 
principles*  expressing  strategic  or  tactical  values  and  ideas  that  are  perceived  to  be  key  factors  in 
successful  restructuring.  These  factors  include  people,  structure,  culture,  and  resources.  However, 
this  value  category  seemed  to  garner  the  least  interest  among  the  study's  main  stakeholders. 


403 


Core  Values 

Core  values  further  embody  seminal  values  and  thus  are  both  illustrative  and  specific 
expressions  of  values.  Core  values  substantially  guide  and  influence  the  choices  and  decisions 
made  in  the  course  of  re-designing  an  educational  system.  Therefore,  they  'enhance  the  creation  of 
the  image  and  the  design  of  the  system11  (Banathy,  1991,  p.  126).  Core  values  can  be  discussed  with 
respect  to  three  different  levels  according  to  their  scope:  the  earning  level,  the  organizational  level, 
and  the  social  level. 

Learning  Level.  This  level  is  comprised  of  stakeholders1  values  that  are  directly  relevant  to 
considerations  of  learning,  which  include  outcomes  of  learning,  process  of  learning,  assessment 
of  learning,  and  environment  of  learning.  The  analysis  above  shows  that  the  study's  main 
stakeholders  believe  that  their  values  regarding  the  learning  level  should  be  considered  and 
articulated  most  strongly  among  the  various  levels  involved  restructuring  their  school  system. 

Organizational  Level.  This  level  is  composed  of  perceptions  relevant  to  the  organizational  level 
-Organizational  Structure  &  Culture  (Nadler,  Gerstein,  &  Shaw,  1992)-  on  which  the 
stakeholders  placed  value  as  an  ideal  for  bringing  out  the  learning  valued  by  themselves. 

Social  Level.  This  level  describes  stakeholders*  perceptions  of  education  on  a  broader  scope, 
the  Function  of  Education  within  the  Larger  Society. 

The  informants  appeared  to  place  priority  on  the  value  areas  directly  related  to  learners  and 
their  benefits.  In  other  words,  it  was  imperative  for  them  to  clarify  and  reach  consensus  on  values 
regarding  who  learners  are,  what  they  should  learn,  how  they  should  learn,  and  how  they  should  be 
assessed.  This  perspective  seems  to  reflect  Banathy  (1991)'s  "a  learning  experience-focused 
system"(p,  115)  in  which  the  learners'  system  is  the  core  around  which  the  system  is  built.  And  the 
stakeholders  placed  much  more  vuluii  on  the  outcomes  of  redesigning  than  on  the  process  uself  or  on 
the  reflection  on  the  rationale  behind  change.  That  is,  it  appeared  that  it  was  more  urgent  to  get  a 
consensus  on  values  regarding  the  nature  of  learners/learning  and  the  ideas  embodying  these 
qualities  which  include  the  outcome***  of  learning,  the  learning  process,  assessment,  and  the  learning 
environment.  General  restructuring  principles,  organizational  structure  and  culture,  and  the 
societal  context  relevant  to  supporting  learning  systems  seemed  to  be  relatively  minor  issues. 

Articulated  Values 

The  values  articulated  and  shared  by  the  stakeholders  appear  to  be  constructed  with  internal 
consistency  with  respect  to  each  other.  Furthermore,  many  of  the  values  and  ideas  were  transferred 
or  fleshed  out  in  other  value  categories.  Thus,  the  informants  appeared  to  understand  the 
interactive  and  interdependent  natures  of  all  the  values  in  the  different  system  levels  and 
dimensions  (Banathy,  1991). 

The  stakeholders  valued  the  recognition,  even  the  celebration,  of  individual  differences  for  a 
variety  of  reasons,  including  learning  styles  and  the  demonstration  of  learning  achievement.  They 
believed  learning  is  a  process  that  makes  sense  of  the  world  around  a  learner. 

The  stakeholders  recognized  the  relationship  between  education  and  the  larger  society.  They 
sought  the  rationale  of  restructuring  in  terms  of  the  gap  between  education's  current  status  and 
change  in  the  larger  society.  In  this  context,  they  emphasized  challenging  obsolete  assumptions 
through  a  continuous  learning  and  change  process.  To  them,  the  fundamental  focus  of  restructuring 
should  be  'each  learner'  and  her  or  his  learning  achievement.  To  achieve  this  goal,  they  believed 
restructuring  should  produce  changes  in  peoples'  mindsets  and  attitudes  toward  the  nature  of 
learners  as  well  as  change  in  structure. 

Regarding  strategies,  the  stakeholders  emphasized  the  'readiness  of  people'  in  the  school. 
Professional  development  and  staff  self-efficacy  were  outstanding  values.  Collective  wisdom  and 
support  availahle  through  divergent  stakeholder  groups  was  considered  to  be  mandatory.  They 


believed  streamlined  communication  to  be  critical  for  effective  communication  within  and  among 
these  groups.  Moreover,  they  believed  restructuring  works  well  when  there  is  a  value-laden  change 
process  allowing  the  people  to  have  choices  and  eventually  develop  ownership.  They  were  fully 
aware  of  the  importance  of  resources  from  comprehensive  areas-both  internal  and  external  -to  help 
restructuring  happen. 

The  stakeholders  valued  'a  whole  person/  who  achieves  excellence  in  the  entire  learning  arena; 
'a  life-long  learner*  who  challenges  her  or  his  own  standards.  They  shared  their  ideun  of  tl  learning 
process  under  the  overarching  value,  MThe  learning  process  should  be  consistent  with  findings  about 
the  nature  of  a  learner  and  learning."  Based  on  the  belief  in  assessment  consistent  with  the 
processes  of  learning  used  and  learning  outcomes  expected  at  CC,  they  focused  on  students' 
continuous  progress  and  comprehensive  and  collaborative  assessment  tools.  They  valued  a  learning 
environment  that  promotes  the  richest  opportunities  for  a  child's  success  in  school.  This 
environment  was  expanded  to  include  areas  outside  classroom  walls. 

They  valued  their  school  as  an  organization  in  which  there  was  a  value-laden  decision  making 
structure  and  streamlined  communication  among  and  between  stakeholders.  They  perceived  that  an 
educational  system  should  be  a  center  for  life-long  learning;  to  this  end,  it  should  coordinate  with 
other  systems  in  the  larger  society. 

Again,  the  values  identified  in  this  research  share  commonalties  with  core  ideas  that  Banathy 
(1991)  proposed,  in  light  of  their  challenges  to  the  dominant  traditional  values  and  practices  in  the 
educational  establishment.  Nevertheless,  those  shared  values  in  the  research  site  tended  to 
represent  its  unique  context,  more  or  less  reflecting  the  constraints  experienced  by  informants. 


VII.  Discussions  and  Implications 

This  research  represents  an  effort  to  provide  systemic  guidance  that  emerged  from  the 
synthesis  of  ideal  elements  identified  from  the  study's  research  Hite.  Through  examining  a  real 
world  school's  restructuring  practices,  I  intended  to  advance  theoretical  understanding  in 
articulating  values  with  regard  to  systemic  sellout  rest  met  tiring,  as  well  as  provide  a  systemic  model 
of  values  categories  which  can  help  stakeholders  to  start,  and  continue  dialogue  to  identify  their  own 
values.  By  doing  so,  eventually  the  findings  of  this  research  may  bridge  the  gap  between  the  real 
world,  which  lacks  theoretical  guidance  in  identifying  values,  and  the  academic  arena,  which  needs 
greater  understanding  of  practice. 

The  categories  that  emerged  from  the  resoairh  site  show  numerous  similarities  with 
the  value  categories  suggested  by  Bnnathy(HH)2)  in  terms  of  their  features,  as  well  as 
comprehensively  reflecting  multiple  levels  and  dimensions  relevant  to  educational  system 
design.  Banathy  has  presented  two  different  schemes  of  value  categories  at  different  times. 
In  1991,  he  (p.  126-128)  presented  three  categories  of  values  including  (1)  educational 
functions  and  purposes,  (2)  learner  and  learning,  and  (3)  systems  design  that  will  guide 
change.  In  1992,  Banathy  (p.  32-33)  suggested  that  in  designing  a  new  educational  system, 
we  need  to  explore  essential  values  related  to  the  followings:  (1)  education's  relationship 
with  and  relevance  to  society,  (2)  the  designation  of  its  societal  functions,  (3)  the  kind  of 
learning  to  be  offered  in  an  information  and  knowledge  age,  (4)  the  way  such  learning  should 
be  offered,  and  (5)  the  design  of  its  organizational  forms  and  arrangements.  All  the  elements 
of  the  value  categories  discovered  in  my  study  are  represented  in  either  one  of  Banathy's 
value  schemes. 

Even  though  my  study  was  conducted  in  an  inductive  way  without  having  any  specific 
propositions  to  examine,  the  similarity  between  Banathy's  non-data-bast . »  research  and  my 
empirical  research  indicates  two  things.  First,  Banathy's  value  categories  are  reliable  and 
applicable  even  with  no  supporting  data.  Second,  my  research  resuls  appear  to  be  more 
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generalizable  since  being  confirmed  by  another  researcher. 

In  using  these  values  categories  meaningfully  and  effectively  in  a  design  process,  several 
factors  need  to  be  considered,  mainly  rooted  in  the  nature  of  values. 

Articulating  values  is  simply  one  of  the  important  elements  in  a  design  process.  Accordingly, 
values  or  values  identification  does  not  have  a  value  in  and  of  itself.  Rather  it  is  meaningful  when 
considered  within  the  context  of  a  design  process.  As  Goulet  and  Dolbec  (1991)  state,  the  systems 
design  process  is  purposive  and  deliberate  so  that,  in  order  to  give  direction  to  human  actions, 
values  should  be  explicated  through  statements  of  mission,  goals,  and  specific  or  concrete  objectives. 

Values  are  dynamic  and  emergent  so  that  values  clarification  should  be  an  on-going  process. 
Values,  by  nature,  shape,  change,  and  reconstruct  throughout  time.  Due  to  the  functional 
interaction  of  each  part  in  a  system,  the  parts  arc  designed  with  respect  to  harmony,  which  is 
concerned  with  "the  effect  of  the  interactions  of  the  parts  on  the  whole",  "the  effects  of  the 
functioning  of  the  whole  and  the  interactions  of  the  parts  on  the  parts  themselves",  as  well  as  "the 
effects  of  the  functioning  of  the  parts  and  the  whole  on  the  containing  system  and  other  systems  in 
its  environment"  (Ackoff,  1981,  p.  17). 

Human  beings,  as  one  part  in  a  system,  change  their  value*  and  belief*  based  on  their 
experiences.  Besides,  as  things  evolve,  more  stau'hohlel'N  and  new  Ntukcholder*  arc1  engaged  in  the 
design  process.  Then  perhaps  different  values  evolve  because  of  the  presence  of  new  people.  As  a 
result,  values  clarification  is  not  a  "one-shot"  activity  or  u  step  in  the  design  process,  hut  is  inevitably 
a  continuous  reflection  and  reconstruction  process  while  stakeholders  and  nrguniKtitionH  proceed 
toward  change. 

There  is  a  possibility  of  espoused  advocated  values  in  organisations  so  that  values  are  not 
consistent  with  individual  and  organizational  actions.  With  this  in  mind,  we  should  be  aware 
of  the  importance  of  a  "designing  culture"  which  promotes  thorough  dialogue  about  personal  values. 
An  organization-the  individuals  in  the  organization-  should  gain  the  ability  to  surface  individual 
values  and  share  them  publicly  and  clearly  through  meaningful  dialogue  throughout  an 
organization. 

All  of  the  issues  discussed  above  lead  to  numerous  questions  to  be  examined  further,  which  can 
not  be  separated  from  using  the  value  categories  above.  Some  examples  might  include:  When 
should  stakeholders  publicly  clarify  values,  so  as  to  be  sources  for  creating  a  new  educational  system, 
not  for  maintaining  the  current  one?  Mow  should  facilitators  or  change  agents  help  stakeholders 
dialogue  on  those  values  areas  and  arrive  at  consensus?  What  would  be  the  best  way  to  translate 
values  into  missions,  visions,  or  specific  goals?  What  does  a  continuous  process  of  value  clarification 
mean?  How  can  we  make  continuous  values  clarification  happen? 
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Appendix  A:  Interview  Questionnaire, 

background  information 

•  In  which  year  did  your  school  begin  it h  nil  rent  rest  met tiring  effort? 

•  What  were  the  forces  driving  or  stimulating  to  these  change  initiatives? 

•  Who  were  the  individuals  or  groups  initiating  the  change?  Do  you  think  they  were  the  best  ones? 
Why?  Why  not? 

•  Who  are  the  groups  or  individuals  driving  the  continuous  change  process?  Do  you  think  they  are 
the  best  ones?  Were  there  any  missing  groups  or  individuals?  Why?  Why  not? 

•  What  was  your  role  in  the  early  change  process? 

•  Mow  do  you  define  your  current  role  in  redesigning  your  school  prognm?  Why? 

•  What  were  the  previous  main  focuses  and  what,  are  the  current  ones?  Why  should  it/these  have 
been  prioritized  more  than  any  others  in  n  specific  period? 

■  In  which  category  is  your  school?  --urban,  suburban,  and  rural? 
And  what  are  the  determiners? 

vf 

•  How  did  your  school  examine  the  beliefs/values  and  develop  this  statement?  Please  tell  me  who 
was  involved  in  it,  when  it  was  written,  and  how  it  proceeded?  Do  you  think  your  school  chose  the 
best  way  to  do  it?  Why?  Why  not? 

•  What  is  the  role  of  a  value  statement  in  redesigning  your  school?  Do  you  think  the  current  belief 
statements  have  responded  well  to  your  purpose?  Why?  Why  not? 

•  In  which  ways,  if  any,  do  you  think  your  school  can  clarify  its  values  better  and  develop  better 
value  statement?  What  will  be  the  contents  of  your  different(revised)  value  statement? 

•  What  are  the  values/beliefs  behind  each  of  these  "Exit  Outcomes"?  That  is,  why  have  these 
outcomes  been  considered  to  be  important  ? 

■  What  are  the  values/beliefs  behind  each  of  the  "Long-Term  Goals"?  That  is,  why  did  you  choose 
these  as  long-term  goals? 

•  What  are  the  values/beliefs  behind  each  of  the  "Intermediate  Goals  1991-92"?  Why  do  you  believe 
each  of  these  goals  are  important? 
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•  Why  are  these  achievements  during  Hfhoo)  year  1991-92  considered  significant? 

•  What  are  the  values/belief  behind  the1  plans/immediate  goals,  programs,  strategies  for  1992-93? 
Why  did  you  decide  on  thi  goals? 

•  What  do  you  envision  the  ideal  image  of  your  Hfhool  to  bci?  ■•  learning  activities,  instructional 
strategies,  programs,  assessment,  or  learning  environment, 

•  How  do  you  define  'learning'?  And  why  do  you  think  so? 

•  What  do  you  believe  about  how  a  student  ieunih  and  thinks?  Why? 

•  What  types  of  learning  and  instruction  does  your  school  have?  How  do  you  feel  about  these? 
What  other  learning  and  instruction  do  you  want  your  school  to  have?  Why? 

•  What  do  you  expect  the  students  to  learn  or  be  able  to  do  when  they  leave  the  school?  Why? 

•  What  are  the  programs  and  learning  activities  to  support  students  in  achieving  these  outcomes  or 
capabilities?  How  do  you  feel  about  these?  Why? 

•  What  are  the  teaching  techniques  and  resources?  How  do  you  feel  about  these?  Why?  Why  not? 

•  How  does  your  school  evaluate  student  achievement?  How  do  you  feel  about  this?  Why? 

•  What  is  the  learning  environment  supporting  these  learning  activities  and  programs?  How  do  you 
feel  about  these?  Why? 

•  If  any,  what  else  do  you  want  to  see  at  your  school  in  terms  of  learning  and  instruction?  Why? 

•  What  is  the  ideal  role  and  responsibility  of  each  stakeholder  you  want  to  see  implemented  at  your 
school?  Why? 

•  What  are  the  governing  structures  that  you  want  to  see  implemented  at  your  school?  Why  do  you 
want  to  have/develop  these  features  of  governance? 

•  What  are  the  informal  or  social  structures,  relationships,  or  communication  channels  between  and 
among  all  stakeholders:  teaching  staff,  administrative  staff,  students,  their  parents,  and  the 
community.  Why  do  you  highlight  this  feature  of  relationship? 

•  What  do  you  think  is  education  functioning  for  learners  and  your  school  community?  Why  should 
this  aspect  be  pursued? 

•  What  are  the  resources  you  think  to  be  important  to  achieve  learning?  Why  are  they  important? 

•  What  are  the  ideal  relationships  between  your  school  and  other  schools  in  the  County?  Why  do 
you  think  these  relationships  should  be  pursued? 

•  What  are  ideal  relationships  that  your  school  envisions  with  other  communities  (e.g.,  business, 
governmental  groups,  etc.)?  Why  do  you  think  there  should  be  this  sort  of  relationship? 

•  What  about  the  ideal  relationship  with  universities?  Why  should  thiH  he  pursued? 

•  What  should  be  the  relationships  with  schools  in  other  states  and  countries?  Why  do  you  think 
this  relationship  is  important? 

•  What  about  the  relationship  with  governmental  or  policy  making  groups?  Why  should  it  be 
pursued? 

•  What  are  the  main  decisions  and  implementations  your  school  has  effected  since  restructuring 
ideas  were  initiated? 

•  If  you  were  asked  to  summarize  the  core  attributes  of  your  school's  change  process,  what  would 
you  put  in  this  list  of  attributes?  Why  do  you  think  your  school  buys  into  each  of  these  change 
approaches? 

•  Who  has  been  involved  in  each  of  the  school's  restructuring  processes?  How  have  they  been 
involved  in  the  change  process?  What  is  the  main  rolo  of  each  of  the  ntukeholders? 

•  What  do  you  think  about  each  stakeholder  group's  (differential)  role  in  the  change  or  decision- 
making process?  Are  they  appropriate?  If  so,  why?  If  not,  why? 
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If  tiny,  how  (in  ymi  think  ymir  wlinol  rnilld  do  hotter  in  terms  of  obtaining  people's  opinions  about 
education?  Why? 

•  What  committees  or  tirgiiiii/etl  groups  tines  your  nt'huol  have  in  order  to  develop.  Do  you  think  it  is 
important  for  your  nrhool  to  have  formal  committees  or  groups?  How  important  is  it?  Why  or  why 
not? 

•  Is  there  any  informal  group  helping  in  redesigning?  What  are  these  groups?  Who  are  involved  in 
these  groups?  Why  do  you  think  these  KfoupH  arc*  useful  in  redesigning  your  school?  Or  why  not? 

•  Does  your  school  have  an  outside  fncilitntor  or  consultant  in  order  to  get  help?  If  so,  who  are  they 
and  what  do  they  do?  How  important  is  it  for  your  school  to  get  outside  (expert)  help?  Why  or  why 
not? 

•  Does  your  school  have  an  outside  experts'  help  in  redesigning  learning  or  instruction?  If  so,  who 
are  they  and  what  have  they  done?  How  important  was  it  for  your  school  to  obtain  their  services? 


•  What  do  you  think  school  restructuring  is? 

•  What  should  school  restructuring  focus  on? 

•  What  do  you  think  are  the  most  difficult  obstacles  to  restructuring  ? 

•  What  do  you  think  are  key  factors  in  successful  restructuring? 

Appendix  B:  The  Summary  of  Contact  Form: 


1.  Pick  out  the  most  salient  points  in  the  contact.  Number  in  order  on  this  sheet  and  note  page 
number  on  which  point  appears.  Number  point  in  text  of  write-up.  Attach  themes  or  codes  or 
aspects  in  CAPITALS.  Invent  themes/codes  where  no  existing  ones  apply  and  asterisk  those. 
Comments  may  also  be  included  in  double  parentheses. 

PAGE 

7.  1.  Students  at  CC  are  expected  to  learn  to  respect  self,  others,  and  the  environment.  I  believe 
that  a  feeling  of  self  respect  and  appreciation  of  others  and  peaceful  environment  around  is  basic  for 
"survival"  as  a  human  being.  PD-LD-OUT/SELF-RESPECT 

7.  2.  Students  at  CC  are  expected  to  learn  to  communicate  effectively.  Here  the  idea  of 
communication  is  very  broad,  including  verbal  and  nonverbal.  Through  the  understanding  of 
others'  viewpoints,  you  can  grow  and  change  yourself.  The  ability  to  communicate  effectively  gives 
that  power.  PD-LD-OUT/COMMUNICATION 

7.  3.  Students  at  CC  are  expected  to  learn  to  apply  problem  solving  processes.  Life  is  full  of 
problems.  Here  is  the  issue  of  empowerment.  I  can  handle  these  things(problems)  and  then  I'm  an 
empowered  person  and  feel  empowered.  PD-LD-OUT/PROBLEM  SOLVING,  EMPOWERMENT 

7.  4.  Students  at  CC  are  expected  to  learn  to  set  and  meet  high  standards.  When  they  set  ideal 
standards  and  try  to  arrive  there,  these  standards  are  helping  them  stretch  and  grow.  Accepting 
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the  rlnilU'iiui'  i"  import nnt  lor  growing.  The  idea  of  Hotting  goals  lor  yourself  mid  imlling  yourself 
tip  to  IiImIh-i  level  ih  Important  for  growing,  PD-LIJ-UUT/IIII III  STANDARDS,  C!IIAI J«KNCIK. 
UlinWINll 


V  h.  Students  at  ('(!  are  exported  to  learn  to  participate  an  a  lifelong  learner  productively  and 
responsibly  in  a  rapidly  changing  world.  Ideas  here  again  involve  growing.  You'll  never  know 
everything.  You'll  never  he  done  learning,  You  are  not  Htatic  and  yon  are  hotter  off  being  an 
informed  portion.  In  a  democracy,  you  are  contributing  by  being  knowledgeable.  PD-LD- 
OUT/LIFELONU  LEARNER,  GROWING 

7.  6.  The  most  appropriate  level  for  the  parents  to  be  involved  is  at  the  instructional  level.  PD- 
ORG  STRUC/ROLES  OF  PARENTS 

7.  7.  We  have  a  responsibility  to  educate  parents:  set  up  the  meeting  and  explain  to  them  the 
aspects  of  programs.  PD-ORG -STRUC/ROLES  OF  SCHOOL  FOR  PARENTS 

7.  8.  Developing  trust  between  teachers  and  parents  is  also  important.  PD-ORG  CUL/TRUST, 
OPENNESS,  PRGU1D/TRUST  BUILDING 

9.  9.  The  quality  of  teaching  is  enhanced  by  implementing  a  staff  development  plan  that  has  focus 
and  continuing  emphasis.  The  undergirding  belief  here  is  the  idea  of  continuous  learning  from  each 
other.  Share  is  one  aspect.  It  is  a  ridiculous  idea  that  teachers  are  educated  and  get  out  of  college 
and  i.re  done  learning.  Learning  has  to  be  continuous.  PR-GUID/STAFF  DEVELOPMENT, 
CONTINUOUS  LEARNING,  SHARING.  PD-ORG-CUL/CONT1NUOUS  LEARNING 

9.  1 0.  Program  effectiveness  is  improved  by  increased  opportunities  for  communication  between 
staff:  .hi at  sharing  ideas  can  charge  people  up.  If  not,  we  can  be  very  isolated.  Through 
fomimi ideation  with  others  we  can  take  the  best  ideas  and  use  them.  Also  you  can  iron  things  out, 
like  misunderstandings,  There's  like  a  little  community.  PD-ORG-CUL/COMMUNICATION 

9.  11.  (letting  reward  from  colleagues  and  others  is  important  since  this  is  a  human  need  and 
professional  need.  PD-ORG  CUIVRECOGN1TION 

9.  1 2,  Authentic/alternative  assessment  of  student  achievement  and  their  program  is  essential  to 
an  instructional  program.  It  is  essential  because  not  everyone  fits  the  same  mold.  Kids  learn  in  a 
lot  of  different  ways  and  are  able  to  show  you  in  a  lot  of  different  ways  what  they  learn.  It's  a  real 
discrimination  just  to  use  only  standardized  tests  to  make  judgment  about  kids.  PD-LD- 
ASS/ALTERNATIVE,  MULTIPLE  WAYS,  PD-LD-LEARNER 

11.  1.3.  Reading  is  the  basis  for  achieving  any  kind  of  significant  learning.  When  they  are  secure  in 
reading,  they  love  to  read.  Students  need  these  kinds  of  fundamentals  in  developing  self-esteem: 
For  the  kid  who  can't  read,  it  is  very  hard  for  them.  PD-LD-PRO/READING,  PD-LDOUT/SELF- 
ESTEEM 

11.  14.  Math  is  connected  with  daily  life.  But,  then  thinking  in  mathematical  way  is  really 
problem  solving.  PD-LD-PRO/MATH,  PD-LD-OUT/PROBLEM  SOLVING 

11.  15.  We  need  to  maximize  the  effectiveness  of  available  technology  as  an  instructional  tool: 
Children  need  to  feel  comfortable  with  any  tool  which  is  available.  I  see  technology  as  a  broader 
way  of  interacting  with  the  world:  a  tool  to  connect  my  classroom  to  the  outside  world  and  to 
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different  cultures.  We  are  relatively  homogeneous.  Here  is  the  idea  of  global  community.  Also  we 
need  to  teach  them  to  understand  and  use  technology  since  it  will  he  a  part  of  their  future  careers. 
PD-LD-PRO/RESOURCES-TECHNOLOGY 

13.  16,  Designation  as  Indiana  2000  gave  us  a  step  of  approval  from  thi!  state.  So  politically  good, 
And  it  gave  us  money  so  we  could  do  staff  development.  It  also  brought  uh  network  with  other 
people  in  Indiana  2000.  Through  the  network,  we  can  support  each  other.  I  think  recognition  is  the 
main  thing.  PR-GUID/LEGITIMIZE,  RECOGNITION,  MONETARY  RESOURCE,  NETWORK. 

13.  17.  We  moved  forward  in  development  of  outcome-based  goals  for  instructional  programs:  An 
outcome-based  program  is  the  thinking  of  what  we  want  a  child  to  be  able  to  do.  It  is  also  that  CC 
can  do  better  for  them  when  they  leave  the  E  county  schools.  It  allows  us  a  cumulative  instruction 
and  learning  and  the  same  direction.  PD-LD-PRO/OUTCOME  BASED  INSTRUCTION, 
CONSTANCY  AND  CLARITY, 

13.  18.  We  implemented  performance  assessment  for  student  progress:  It  seems  a  much  less 
artificial  way  of  assessing  given  tasks.  If  I  see  there's  value  to  something  I  understand  and  where 
I'm  going,  and  if  I  get  proof  of  what  I  can  do,  then  there  is  a  lot  of  satisfaction.  PD-LD-LEARN,  PD- 
LD-ASS 

14.  19.  We  implemented  continuous-progress  instructional  program  at  the  primary  level: 
Children  don't  learn  exactly  in  nine  months  and  nine  months.  There  is  no  limit  to  how  far 
they  ought  to  go  or  they  can  go  in  that  period  of  time  and  so  get  the  child  moving  along  their 
path.  And  if  they  really  satisfy  the  requirements  at  their  grade  level,  then  why  not  let  them 
go  on  and  explore  more.  That's  what  the  continuous  program  is  about.  They  should  go  where 
they  need  to  go  and  it  is  not  determined  by  the  grade  level.  PD-LD-PRO/CONTINUOUS 
PROGRESS,  PD-LD-LEARN/CONTINUOUS  LEARNING 

14.  20.  I  think  it  is  really  important  that  individuals  arc  allowed  to  grow  at  their  speed 
whatever  their  speeds  are.  But  we're  tied  together  through  thematic  instruction  so  we  are  all 
studying  the  same  things  but  the  levels  of  what  is  dealt  with  are  different.  I  think  thematic 
instruction  is  a  good  balance  since  we  need  to  share  the  experiences.  Community  in  a 
classroom  is  very  important  because  we  are  looking  at  the  same  theme.  What  teachers  and 
students  do  with  these  themes  could  be  more  individualized.  PD-PD-PRO/THEMATIC 
INSTRUCTION,  SHARING  EXPERIENCES 

14.  21.  What  will  be  the  best  way  to  restructure  ? 

>  You  have  to  bring  people  along  and  have  to  have  support  but  you  can't  make  any  change 
into  the  next  century  through  step  by  step.  PR-GUID/??? 

14.  22.  In  which  way  is  your  school  restructuring? 

>  I  think  we  have  one-three-five  year  plans.  Some  of  the  changes  are  rapid,  such  as  the 
ungraded  classroom  structure,  but  it  wasn't  for  everyone.  So  you  could  do  some  rapid  things 
as  long  as  there  are  some  choices  people  can  be  involved  in.  You  don't  make  anybody  to  do  it. 
It  is  always  voluntary  here.  Maybe  we  reassess  what  we  thought  we  were  doing.  So  I  think 
we  should  do  step  by  step  planning  but  at  the  same  time  something  should  be  brought  along 
relatively  quickly  if  you  have  support.  Then  people  who  feel  uncomfortable  with  change  will 
have  a  place  to  be  rather  than  just  complain.  It  was  always  either  or  or.  I  think  more  and 
more?  teachers  are  involved  in  multi-age  grouping.  PR-GUID/CHOICE,  SCHOOLS  WITHIN 
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15.  23.  It  seemed  very  logical  to  outreach  since  we  have  a  lot  of  strength  in  faculty.  Around 

75  %  of  the  faculty  feel  comfortable  and  are  willing  to  present  their  expertise  at  conferences. 

We  have  huge  numbers  of  volunteers.  We  had  organizational  ability  and  we  could  take  it. 

We  are  willing  to  go  beyond  and  share.  Also  it  is  here  the  issue  of  pride  in  what  aro  doing 

and  opportunity  show  up.  PR-G,U1D/SHARING,  USING  STRENGTH,  RECOGNITION, 

PRIDE,  FEELING  OF  EFFICACY,  VOLUNTEERS 
t 

15.  24.  We  continued  as  National  Reading  Styles  Institute  Model  School  Site:  NRSisakind 
of  organized  way  of  connecting  learning  styles  (individual  differences  and  preferences)  to 
reading.  This  is  one  way  to  remove  some  of  the  roadblocks.  PD-LD-PRO,  PD-LD- 
LEARN/INDIVIDUAL  DIFFERENCES  IN  LEARNING  STYLES  AND  PREFERENCES 

16.  25.  As  faculty,  we  thought  it  would  be  good  that  student  teachers  are  involved  in  multi- 
aged  groups  but  nobody  was  interested  in  it.  We're  trying  to  find  connections  with  an 
university  in  some  other  ways.  I  know  we  can  support  each  other.  What  we've  been  doing 
here  is  based  on  whole  language  and  literature  based  methods.  I  hope  for  a  professional 
relationship  with  AN  UNIVERSITY. 

PR-GUID/RELATIONSHIP  WITH  SOCIETY 

17.  26.  We  created  opportunities  for  children  in  grades  K-6  to  participate  in  life  skills-based 
economies  education  project:   It  gives  kids  a  chance  to  do  something  having  a  sense  of  real 
life.  We  need  to  connect  more  and  more  with  children's  real  life.  Unless  kids  feel  that  is 
important,  saying  that  this  is  important  does  not  work.  Technology  should  be  a  tool  for 
exploring  the  real  world.  PD-LD-PRO/REAL  LIFE,  PD-LEARNING  ENV/TECHNOLOGY 
AS  TOOL 

17.  27.  We  actively  promoted  and  established  after-hours  use  of  the  building  as  a 
community-based  resource  for  children:  Parents  were  really  interested  in  the  school  as  more 
of  a  community  center.  By  having  social  services  in  the  building  and  also  using  the  building 
in  the  evening,  it  gives  people  a  place  to  belong  to  and  a  feeling  of  place  for  the  community.  I 
personally  like  more  and  more  moving  toward  that  direction.  PD-SOCIAL- ED/SCHOOL  AS 
COMMUNITY  CENTER 

17.  28.  We  developed  building-wide  awareness  for  a  thematic  approach  to  instruction: 
We've  been  reading  a  lot  about  brain  research  and  about  the  ways  people  think.  Based  on 
what  we  know  about  how  kids  are  learning,  they  do  not  learn  in  separate  subjects,  but  within 
a  context.  PD-LD- PRO/THEMATIC  APPROACH,  PD-LD-LEARN  *VALUES/CONSTANCY 
BETWEEN  VALUES. 

18.  29.  The  immediate  goal  (92-93)  is  the  development  of  an  instructional  program  which 
has  clarity  and  consistency  throughout  the  age  levels  of  school:  When  programs  have  clarity 
and  consistency,  students  do  a  bettc  "  job  because  they  are  comfortable  with  it.  Even  maybe 
teachers'  personalities  are  very  different.  Maybe  or  maybe  not  this  new  teacher  will  use  the 
thematic  approach.  But  the  program  at  different  levels  shouldn't  be  totally  different  from 
the  last  year,  shouldn't  cost  last  year's  experiences.  Clarity  and  consistency  in  an 
instructional  program  help  kids  feel  comfortable  with  the  learning  process.  PD-LD- 
PRO/CLARITY  AND  CONSTANCY,  PD-LD- LEARN/SELF-COMFORT 

18.  30,  We  offer  the  Up- Lift  program  as  an  option.  Changing  is  uncomfortable  for  a  lot  of 
people,  but  I  think  it's  good.  I  think  w>nu»  of  the  parents  still  might  be  uncomfortable.  For 
nehool,  ntill  to  some  extent  there  is  a  need  to  push  people;  that  is  DF's  job.  He  keeps  the 
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whole  picture.  He  1ms  a  kind  of  balance  -  supporting  people  trying  innovation  but  also  not 
loosing  people  who  ieel  uncomfortable.  PR-GUID/CHOICE,  VOLUNTARY  PARTICIPATION 


Appendix  C:  A  START  LIST  OF  CODES 


PRODUCT  OF  RESTRUCTURING  PD  1 


LEARNING  DIMENSION  PD-LD  1.1. 


PD-LD  Nature  of  Learners  &  Learning  PD-LD-LEARN  1.1.1. 

PD-LD  Outcome  of  Learning  PD-LD-OUT  1.1.2. 

PD-LD  Process  of  Learning  PD-LD-PRO  1.1.3. 

PD-LD  Assessment  PD-LD-ASS  1.1.4. 

PD-LD  Learning  Environment  PD-LD-ENV  1.1.5. 


ORGANIZATIONAL  DIMENSION  PD-ORG  1.2. 


PD-ORG        Organizational  Structure  PD-ORG-STRUC  1.2.1. 

PD-ORG       Organizational  Culture  PD-ORG-CUL  1.2.2. 


SOCIAL  DIMENSION  PD-SOCIAL  1.3. 


PD  ■  SOCIAL  Role/function  of  education 

PD-SOCIAL-ED  1.3.1. 

PD-SOCIAL   Role/function  of  society 

PD-SOCIAL-SO  1.3.2. 


PROCESS  OF  RESTRUCTURING  PR  2 


PR  General  Principles  PR-GEN  2.1 

PR  Guiding  Principles  PR-GUID  2.2 


SITE  INFORMATION  INF  3 


INF             DEMOGRAPHICS                          INF- DEMO  3.1 

INF             EVENT  CHRONOLOGY                   INF-CHRON  3.2 

INF             STRUCTURE                               INF-STRUC  3.3 

INF             CULTURE                                   INF-CUL  3.4 

Appendix  D:  DEFINITIONS  OF  CODES 

1.  PRODUCT  OF  RESTRUCTURING 
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1.1.  Learning  Dimension 


1.1.1.  Nature  of  Learner/Learning: 

Basic  assumptions  or  values  which  affect  the  articulation  of  the  other  values  or  ideas, 
especially,  relevant  to  process  of  learning  and  assessment 

>  Nature  of  Learner:  values  or  id^as  about  the  ways  that  a  learner  learns  the  best; 

values  or  ideas  about  the  natures  of  a  learner  including  intellectual,  social, 

emotional,  and  physical 

>Nature  of  Learning'  definition  of  learning,  focus  of  learning,  climate  for  learning 

1.1.2.  Outcomes  of  Learning/Learning  Outcomes:  what  the  learner  will  be  able  to  know 
and  do  after  the  process  of  learning. 

1.1.3.  Process  of  Learning/Learning  Process:  values  and  ideas  regarding  how  to  achieve 
learning  outcomes.  That  is,  the  ways  of  learning  including  the  types  of  learning 
arrangement  or  strategies  and  learning  situations.  They  should  be  consistent  with  the 
values  concerning  the  nature  of  learning  and  learner;  and  they  should  promote  the 
learner  to  achieve  the  expected  learning  outcomes. 

1.1.4.  Assessment:  values  or  ideas  in  regard  to  the  purpose  of  evaluation  (why),  the 
content  of  evaluation  (what),  the  way  of  evaluation  (how),  time  of  evaluation  (when), 
and  party  (ties)  of  evaluation  (who).  They  should  be  consistent  with  the  values 
concerning  the  learning  outcomes  as  well  as  the  nature  of  learning  and  learner. 

1.1.5.  Learning  environment:  values  about  the  ideal  environments  which  might 
support  the  process  of  learning  that  is  designed  to  ensure  learners  to  achieve  the 
expected  learning  outcome:  physical  and  human  resources;  social,  cultural,  and 
disciplinary  atmosphere. 

1.2.  Organizational  Dimension 

1.2.1.  Organizational  structure:  Formal  organizational  arrangement  (Nadler.  Gertwoin, 
&  Shaw,  1992,  p.  133):  a  governance  structure,  formalized  roles  and  responsibilities  of 
stakeholders,  and  organizational  communication  channels/bodies. 

1.2.2.  Organizational  culture:  I  mean  'organizational  culture1  in  this  definition  only  the 
highest  level  of  Schein's  model  (Nadler,  Gerstein,  &  Shaw,  1992,  p.  134):  visible 
behaviors,  or  'artifacts1,  in  a  daily  life.  These  include  manner  of  speaking  among 
stakeholders,  standardized  patterns  of  behavior,  expression,  gesture,  events,  and 
rituals. 

1.3.  Social  Dimension 

1.3.1.  Rolc/funct  ion/pur  pose  of  education:  values  and  iduaa  about  tin?  function  of 
education,  the  relationship  of  an  educational  system  with  other  social  organizations. 

1.3.2.  Uole/function  of  society:  an  educational  system's  expectations  toward  other  social 
organizations;  fundamental  relationships  of '.he  larger  society  with  an  educational 
system. 
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2.  PROCESS  OF  RESTRUCTURING 


2.1.  Genera]  Principles 

General/fundamental  principle  of  restructuring:  reasons  tor  why  restructuring  is 
needed,  the  definition/nature  of  restructuring,  and  the  main  focus(es)  of  restructuring. 

2.2.  Guiding  Principles 

Strategic  or  tactical  values  or  ideas  which  include  internal  factors  and  external  key 
factors  to  success:  people  (who  should  be  involved);  process  (how  should  restructuring 
be  processed;  governance/organizational  structure;  organizational  culture/working 
relationships,  internal  and  external  resources  (time,  technology,  people,  monetary....) 

Appendix  E:  Interview  Summary  Report 
C,  MM. 

First  Interview. 

1.  1.  The  biggest  thing  to  me  and  many  of  my  peers  is  collaborative  staff.  We  feel  very 
comfortable  about  asking  to  be  taught  by  others  that  have  expertise.  We  also  feel  comfortable  with 
offering  to  share  over  the  course  of  time. 

1.  2.  No  one  individual  knows  or  can  lead  the  change.  Depending  upon  their  expertise, 
people  are  concentrated  on  certain  areas  based  on  collaboration. 

2.  3.  I  feel  comfortable  that  any  parent  of  my  students  calling  or  talking  with  DF.  Since  I 
thir  k  she  might  need  to  know  what  he  knows  better  than  I  do  or  needs  something  ho  can  provide 
better  than  I  do. 

2.  4.  We  are  interested  in  students  seeing  computers  as  tools:  utilize  computers  as  another 
tool  in  their  learning  process. 

3.  5.  We  need  to  provide  what  they  think  is  important.  Also,  We  need  to  provide  education 
in,  not  just  a  competitive,  but  a  cooperative  and  social  context. 

3.  6.  We  try  to  teach  students  to  actually  demonstrate  what  they  karn  at  CCt  we  do  not 
want  them  io  leave  with  just  report  cards  with  happy  faces  and  pluses  and  checks.  We  want  the 
children  to  be  able  to  gc  out  to  the  real  world  and  demonstrate  the  ability  they  learn  to  communicate 
by  reading,  writing,  speaking,  and  listening. 

3.  7.  We  don't  do  the  same  things  better,  we  do  not  do  completely  different  things  either,  by 
throwing  away  old  things.  But  we  borrow  from  effective  school  programs,  effective  leadership 
models. 

7.  8.  In  our  culture,  we  don't  respect  others  because  we  were  not  truly  taught  to  respect  oui 
own  individual  uniqueness.  Respecting  '  ie  environment  is  becoming  a  more  and  more  obvious  is^ue. 
So  we  need  to  teach  our  kids  to  respect  self,  others,  and  the  environment. 

7.  9.  If  families  and  others  can't  provide  these  kinds  of  things:  respect  self,  others  :  id  the 
environment,  there  is  a  need  that  we  (schools)  have  to  provide  this. 

7.  10.  Communicate  effectively.  Preparing  the  students  for  the  adult  world-jobs.  That  is 
one  issue  of  communication.  Children  as  well  as  staff  need  to  learn  all  of  those  critical 
communication  skills. 

7.  11.  Reeding  is  a  way  of  gleaning  information.  Reading  allows  us  to  understand  what  the 
author  of  that  material  wants  to  communicate. 

7.  12.  Students  need  to  learn  in  context. 

fi.  13.  The  world  is  always  rapidly  changing  becaune  we're  iii  the  information  ,ige  in  which 
there  is  a  dramatic  impact  of  technology.  We  need  to  teach  the  children  to  recognize  problems  and 
use  their  own  nwurcen  fl»  a  "lifelong  learner". 
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8.  14.  Student  should  know  to  set  and  meet  high  standards.  Each  student's  high 
expectation  is  unique  to  each  student.  We  as  a  teachers  should  help  them  to  constantly  assess  and 
reset  new  goals. 

8.  15.  Students  learn  to  participate  as  a  lifelong  learner  productively  and  responsibly  in  a 
rapidly  changing  world. 

9.  16.  Because  the  world  is  dramatically  changing,  the  population  is  changing,  group 
dynamics  are  changing  in  the  classrooms,  strategies  can't  be  the  same.  We  are  open  to  that  change 
and  open  to  that  communication  which  will  take  place. 

9.  17.  Our  society  is  less  than  functional  because  we  have  such  a  competitive  bend,  not 
cooperative  and,  thus,  we  are  not  served  to  a  useful  end. 

10.  18.  The  quality  of  teaching  is  enhanced  by  implementing  a  staff  development  plan  that 
has  focus  and  continuing  emphasis. 

10.  19.  Program  effectiveness  is  improved  by  increased  opportunities  for  communication 
between  staff. 

1 1.  20.  Alternative  assessment  of  student  achievement  and  programs  is  essential  to  an 
instructional  program.  For  students,  it  allows  them  to  analyze  themselves  and  allows  the  class  to 
review.  How  they  feel  about  themselves  is  far  more  important  and  how  their  peers  feel  about  them  is 
far  more  important  than  what  I  feel  about  them. 

12.  21.  For  students,  reading  can  be  seen  as  one  of  the  easiest  and  clearest  manifestations  of 
communication  events.  Also,  it's  a  tool  for  respecting  yourself.  If  you  feel  you  can't  do  something, 
you  never  attempt  to  do  it. 

12.  22.  Our  life  depends  upon  mathematical  concepts. 

12.  23.  In  our  society,  computer  aKsists  our  life  in  many  ways.  Computers  and  technology  is 
in  all  aspects  of  our  life  whether  we  want  it  or  not,  whether  we  realize  it  or  not.  We  need  to 
maximize  the  effectiveness  of  available  technology  as  an  instructional  tool. 

14.  24.  Our  designation  hh  an  Indiana  2,000  school  site:  It  gave  us  credence.  We  are  not 
anymore  seen  as  risk  takers  out  on  the  edge.  That's  recognition  for  what  you  have  done.  It 
(officially)  opened  the  opportunity  to  invite  people  in  our  building  and  share.  One  of  the  most 
important  ways  you  learn  something  is  to  teach  someone  else.  Monetary  rewards  allowed  us  to  do 
more.  Usually  we  as  teachers  have  to  do  more  with  less.  This  allowed  us  to  do  more  with  more. 

14.  25.  Without  continuous  progress  types  of  thought  within  classroom  teachers'  strategy 
and  instruction  plans,  a  student  will  not  have  the  ability  to  progress  at  his  continuum,  not  according 
to  the  school  continuum.  If  we  don't  teach  the  way  the  children  learn,  we  will  teach  the  ways  they 
learn. 

15.  26.  We  hosted  our  own  conference  in  which  we  invited  people  to  see  specific  elements  of 
what  we're  doing  at  CC,  what  we  think  is  unique  about  CC  and  maybe  unique  for  E  County.  We  did 
not  want  to  be  seen  as  blowing  our  horns.  We  wanted  to  take  pride  in  what  we  are  doing  and 
communicate  it  with  other  learning  environments.  We  believe  what  we're  doing  can  be  transferred 
to  other  environments. 

15.  27.   We  developed  an  extra-curricular  instrumental  band  for  ten  and  eleven  year  old 
children.  Some  students  can  excel  in  academics  and  others  in  sports.  We  should  give  them  the 
opportunities:  in  which  they  feel  self  actualization,  in  which  to  feel  comfortable,  in  which  to  feel 
growth,  in  which  they  feel  regard  for  their  own  abilities. 

16.  28.  We  created  opportunities  for  children  in  grades  K-6  to  participate  in  a  life  skills- 
based  economies  education  project.  Why  is  that  important  to  the  children  to  learn  these  things? 
Part  of  it  goes  back  to  communication.  What  we  do  in  CC  goes  to  the  idea  of  life  long  learning. 

16.  29.  We  actively  promoted  and  established  after-hours  use  of  building  as  a  community- 
based  resource  for  children.  Then?  is  a  great  need  in  our  community  for  children  after  school  to 
practice  sports,  to  be  engaged.  It  also  gives  positive  ways  for  adults  to  interact  with  their  kids  as 
well  as  others.  Because  we  are  a  building  with  public  money  it  should  be  open  to  the  public  for  their 
use. 
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16.  30.  We  developed  a  building-wide  awareness  for  the  thematic  approach  to  instruction. 
That's  because  the  thematic  approach  gives  children  a  framework  on  which  to  learn  content  and 
insights  into  the  others1  schema.  Without  it,  everything  we  learn  is  just  an  isolated  bit  of 
information.  Without  some  kind  of  connection  or  integration,  we  can't  be  as  effective  in  school  as  we 
should  or  could  be.  Content  is  not  divorced  from  process,  process  of  understanding  the  content  is  not 
divorceu  from  the  product  of  the  process.  They  are  not  necessarily  ends  and  means  of  one  another. 

16.  31.  We've  focused  on  the  development  of  an  instructional  program  which  has  clarity  and 
consistency  throughout  the  age  levels  of  school.  If  not,  there  might  be  gaps  in  our  curriculum  within 
this  building. 

17.  32.  We've  been  implementing  a  performance-based  curriculum  model  in  which  multi- 
dimensional assessment  tools  are  used  to  measure  student  progress  and  program  effectiveness. 
Why?  I,  as  an  assessor,  might  be  not  good  at  certain  aspects  of  evaluation.  We  need  multi 
perspectives  and  insights  in  order  to  have  e  balance  in  evaluating  the  students'  achievement. 
Traditionally,  we  have  focused  on  summative  evaluation.  But  formative  evaluation  should  be  more 
focused  because  it  allows  us  to  evaluate  children's  continuous  progress.  These  assessment  tools 
allow  us  to  evaluate  how  they  grow  in  multiple  ways. 

1  7.  33.  Environment  element;  We  are  planning  to  build  the  outdoor  stage  and  sitting  in  the 
woods  outside  the  building.  The  belief  behind  this  is  an  extended  concept  of  classrooms:  Learning 
places  are  not  confined  within  the  classroom's  wall. 
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Appendix  F:  Matrix 


1.1.1.  PD-LD-LEAR 

Interviews 

Nature  of  Learners  and  Learning 

1st 

2nd 

VI 

1st 

2nd 

m 

1st 

m 

2nd 

1 — u 
1st 

2nd 

IV 

1st 

r? 

2nd 

1st  2 

Nature  of  Learner 

•  Differences  as  a  cause  for  celebration.  1-a 

3.1. 

*  Recognition  of  individual  differences. \ -a 

K  I1 

r 

a  n 

•  Kids  learn  in  a  lot  of  different  ways:  they  have  different  modality, 
strength,  and  preference,  1-a 

2.2. 

9.12 

15.1 

4.5. 

4.7. 

4.8. 

7  19 

•  Kids  are  able  to  show  in  a  lot  of  different  ways  what  they  learn.  1-b 

9.12 

173 

•  Children  don't  learn  exactly  in  a  determined  period.  Each  child 
developmentally  reaches  a  certain  pc:nt  with  different  rates  of  time.  1-b 

3J. 

14.1 

* 

7.12, 

*  Kids  do  not  learn  in  separate  subjects,  but  within  a  context.  2 

17,2 

1. 

163 

f. 

4.6. 

•  Students  do  a  better  job  what  they  are  comfortable  with.  2 

18.2 

>. 

15.1 

6.17 

•  In  the  long  term,  investment  to  younger  children  has  greater  impact. 

6.20 

> 

•  A  child  learns  something  is  important  rather  than  when  they  learn. 

11.2 

K 

•  Kids  need  a  sense  of  ownership/responsibility  about  what  they  are  doin* 
;  internal  motivation.  2 

3.1. 

7.9. 

•  Self-esteem  is  basis  for  accomplishment.  2 

11.2 

1 

7.11 

Nature  of  Learning 

•  Learning  is  a  'process';  it  takes  time.  1 

3.6. 

10.1 

16.3 

I 

•  Learning  is  'making  sense  of  the  world.'  2 

3.6. 

14.2 

•  Learning  does  not  stop.  3 

5.17 

7.5. 
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Appendix  G;  Appendix  J:  Primary  vs.  secondary  value  categories  inventory 
Stakeholders'  view  of  primary  and  secondary  value  categories 


1  isted  below  are  1 1  value  categories  identified  from  interviews,  document  analysis,  and  observation 
at  your  school.  Indicate  how  you  feel  about  the  importance  of  each  value  category  (definitions  of 
value  categories  attached)  by  circling  the  appropriate  number  on  a  scale  of  1  to  5. 
A  higher  number  indicates  primary,  ho  that  is  more  important.  A  lower  number  indicates  secondary, 
ho  thai  in  lens  important.  / 


1    Niitnrc  of  Learner  1 

2 

15 

4 

5 

2.  Nut  lire  of  Learning  1 

2 

3 

4 

5 

II.  tleiienil  Principle  of  Restructuring  1 

2 

3 

4 

fi 

•1   1 'rororiurnl  Principles  of  Restructuring  1 

2 

3 

4 

5 

fi  Oiilniini'Hof  1  yearning  1 

2 

a 

4 

5 

(i.  I'iih'i'hh  of  lvearning  1 

2 

3 

4 

5 

7     AHHI'HHIIII'tlt  1 

2 

3 

4 

5 

N  Li'iirniiig  lOnvironment  1 

2 

.3 

4 

5 

tl  ( li'giini/.ntionul  Structure  1 

2 

3 

4 

5 

II).  ( li'Kiiiii/.iitioniil  Culture  1 

2 

3 

4 

5 

1 1    I'uiirtinn  of  Kihuiition  Within  n  Larger  Society  1 

2 

3 

4  " 

5 

I'lense  forward  thin  sheet  to  tin*  following  adilreHH  or  fax  number: 
In-Kook  Lee 
IfUKi  Must  Urd  Street 
ft  10!)  Mas!  University  Apt 
l)l<mmiligtnii,  IN  47'HIH 
FAX)  N12-Mi7-H.mf. 

Appendix  H:  DeflnitloiiN  of  Value  Cuteuorles 

Definitions  of  Value  CutegmieH 

Dear 

Listed  below  are  the  definitions  of  the  value  categoricB  whirh  are  provided  in  order  to  enable  you  to 
have  a  clearer  idea  about  each  term  presented  on  the  opinion  sheet.  Please  rend  these  descriptions 
before  you  answer  the  opinion  sheet,  Stakeholders'  view  of  primary  and  secondary  volue  categories, 

1.  Nature  of  a  Learner: 

values  or  ideas  about  the  way*  in  which  a  learner  learns  the  best;  values  or  ideas  about 
the  nature  of  a  learner  including  intellectual,  social,  emotional,  and  physical. 

2.  Nature  of  Learning: 

values  or  ideas  reflecting  the  fundamental  attributes  and  functions  of  learning. 

3.  General  Principles  of  Restructuring: 

values  reflecting  the  reasons  for  restructuring,  the  nature  of  restructuring,  and  the 
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main  focus (es)  of  restructuring. 

4.  Procedural  Principles  of  Restructuring: 

strategic  or  tactical  values  or  ideas  which  are  perceived  to  be  key  factors  to  successful 
restructuring.  These  factors  include  both  internal  and  external  factors. 

5.  Outcomes  of  Learning: 

values  or  ideas  relevant  to  the  expectations  of  what  a  learner  will  attain  after 
completing  the  process  of  learning. 

6.  Process  of  Learning: 

values  and  ideas  regarding  how  students  should  achieve  the  expected  learning 
outcomes. 

7.  Assessment: 

values  or  ideas  reflecting  the  purpose  of  evaluation  (why),  the  content  of  evaluation 
(what),  the  way  of  evaluation  (how),  time  of  evaluation  (when),  and  party(ties)  of 
evaluation  (who). 

8.  Learning  Environment: 

values  about  the  ideal  environments  which  might  support  the  process  of  learning  that 
designed  to  ensure  that  learners  achieve  the  expected  learning  outcome. 

9.  Organizational  Structure: 

values  and  ideas  describing  formal  organizational  arrangement  which  includes  a 
governance  structure,  formalized  roles  and  responsibilities  of  stakeholders,  and 
organizational  communication  channels/bodies. 

10.  Organizational  Culture: 

values  and  ideas  relevant  to  visible  behaviors,  or  'artifacts',  in  daily  life. 

11.  Function  of  education  within  a  larger  society: 

values  and  ideas  reflecting  the  goals,  functions,  and  responsibilities  of  an  educational 
system  within  a  larger  society  and  relationships  of  the  larger  society  with  an 
educational  system. 
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Key  Elements  in  Successful  Educational  System  Redesign: 
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Introduction 


There  has  been  an  increasing  perception  of  the  need  for  fundamental  redesign  in  the  American 
educational  system.  This  restructuring  movement  has  received  its  primary  impetus  from  the 
realization  that  the  current  system  fails  to  satisfy  individual  learners*  and  societal  needs  as  society 
continues  to  change  (Banathy,  1991;  Reigeluth,  1987;  National  Commission  on  Excellence  in 
Education,  1983;  Walberg,  1984;  Walberg  &  Fowler,  1987;  Branson,  1987).  Not  only  on  the 
national  level  but  also  on  the  state  and  local  levels,  countless  numbers  of  people  have  put  their 
time,  energies,  and  resources  into  redesigning  efforts. 

Despite  these  attempts,  however,  evaluation  of  earlier  restructuring  indicates  little  difference 
in  practice  or  achievement  as  compared  to  previous  years.  The  more  we  realize  efforts  at 
redesigning  have  not  worked  well,  the  more  we  come  to  seek  factors  that  would  more  likely  ensure 
successful  educational  systems  redesign.  Recognizing  the  demand  for  a  knowledgebase  to  enhance 
redesign,  the  current  study  aimed  to  add  to  that  base  by  identifying  factors  that  are  likely  to  lead  to 
successful  redesigning  of  educational  systems. 

Research  Objectives 

1.  To  synthesize  the  key  elements  involved  in  the  successful  redesign  of  educational  systems 
through  literature  review, 

2,  To  uncover  the  key  elements  in  enhancing  success  in  educational  systems  design  by 
synthesizing  datn  from  two  research  sites. 

i\.  To  provide  n  synthesis  of  the  key  elements  gleaned  from  existing  theory  and  the  present 
nine  study. 

1.  Key  KletnentH!  Learning  from  Theoretical  Perspectives 

1  reviewed  theoretical  and  empirical  research  relevant  to  the  factors  affecting  successful  school 
redesigning  ho  as  to  present  a  broad  picture  of  these  various  elements.  The  review  includes  the 
following: 

1.  A  study  of  the  'institutionalization'  of  new  educational  systems,  based  on  a  comprehensive 
analysis  of  institutionalization  in  various  organizational  sectors  and  of  case  studies  in  sciiool 
restructuring  (Miles  nnd  Kkholm,  MM;  Miles,  1993). 

2.  A  theoretical  framework  including  the  key  elements  in  educational  systems  redesign,  based 
on  conceptions  of  the  needs  for  concomitant  consideration  of  the  appropriate  key  elements  within 
educational  systems  and  their  contextual  relationship  with  societal  systems  (Lee,  1992). 

3.  A  study  focused  on  the  major  reasons  of  why  most  previous  restructuring  efforts  have  failed 
(Keefe,  1993). 

4.  A  study  of  educational  system  elements  particularly  important  to  schools,  districts,  and 
states  when  moving  through  the  systems  design  process  (Anderson,  1993). 

5.  A  study  of  the  key  dimensions  involved  in  successful  implementation,  especially  in  light  of 
teachers'  view.  Put  differently,  this  study  revealed  those  conditions  under  which  teachers  tend  to  be 
more  willing  and  able  to  engage  in  the  change  process  (Evans,  1993). 

6.  A  study  regarding  the  primary  features  of  strategies  for  achieving  systemic  redesign  of 
education  (O'Neil,  1993). 

7.  A  study  of  the  support  and  implementation  strategies  identified  at  the  Center  for 
Educational  Technology  (Florida  State  University),  based  on  the  view  that  "Design  requires  that  we 
are  not  constrained  in  our  thinking  by  current  practice  or  by  limitations  which  may  not  be  relevant 
in  a  redesigned  system'1  (Salisbury,  1993,  p.  135). 
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S.  A  case  study  of  a  major  metropolitan  public  school  district  undergoing  a  fundamental 
restructuring  process,  which  identifies  the  main  factors  contributing  to  this  district's  successful 
redesign  process  (Breidenbach,  1989). 

9.  A  nationwide  case  study  involving  diverse  information  about  the  systemic  restructuring 
process  as  well  as  its  products,  including  the  key  factors  related  to  a  successful  change  process 
(Reigeluth,  Norris,  &  Ryan,  1991). 

Findings  from  Literature  Review 

From  the  above  literature  review,  the  following  key  elements  in  a  successful  redesigning  process  can 
be  drawn. 

Visions 

Given  the  large-scale  changes  accompanying  the  re-examination  of  people's  mindsets  and 
values,  it  seems  critical  for  stakeholders  to  clarify  and  share  their  visions,  that  is,  their  views  of 
what  schools  should  be  like.  These  visions  should  be  future-oriented,  not  limited  to  current 
conditions.  Visions  provide  a  foundation  for  ongoing  design  efforts,  by  supplying  direction, 
generating  new  activities,  and  providing  criteria  for  troubleshooting  (Miles  &  Ekholem,  1991,  p.  15). 

Visions  articulated  from  stakeholders'  core  values  are  those  most  likely  to  be  accepted.  In 
addition,  those  visions  should  be  translated  into  communicated  and  measurable  goals. 

Comprehensive  Plan 

The  literature  review  showed  the  importance  of  formulating  visions  into  a  clear,  consistent, 
comprehensive  strategic  plan.  Such  a  plan  represents  "the  incorporation  of  major  and  visionary 
recommendations  into  the  more  specific  goals  or  blueprint"  (Salisbury,  1993) 

Change  Management 

Comprehensive  educational  systems  redesign  requires  long-term  effort  and  commitment. 
Therefore,  the  change  process  should  be  managed  with  willful  and  intelligent  design  strategies  and 
skills.  The  literature  review  specifically  addressed  a  knowledgebase  relevant  to  the  following 
factors:  planned  coordination  between  and  among  all  stakeholders  (internal  and  external), 
management  of  the  pain  that  accompanies  a  long-term  change  process,  needs  analysis  of  individuals 
and  groups  involved  in  the  redesigning  process,  change  eventually  to  be  embedded  into  stable 
organizational  routines,  transition  management,  personal  and  organizational  learning,  and  avoiding 
stateness. 

Resources 

Human  and  physical  resources,  both  internal  and  external,  should  be  fundamental  to  any 
systemic  change.  Intellectual  (instructional  and  organizational  strategies/knowledge),  financial,  and 
technological  resources  are  required  to  initiate  as  well  as  maintain  change.  In  order  to  decrease 
risks  and  enhance  the  quality  of  redesigning  efforts,  multiple  sponsorship  is  recommend  (Salisbury, 
1993). 

Networking 

Networking  is  essential  in  collaborative  redesigning  efforts  and  eventually  is  more  likely  to 
lend  to  establishment  of  lasting  systemic  change.  Through  a  network,  local  schools  can  outreach  to 
collaborate  in  studying,  piloting,  and  supporting  a  new  vision  of  the  educational  system. 

Support 

Success  in  redesigning  a  system  is  mor"  likely  if  n  school  has  strong  external  support.  A 
significant  aspect  of  systemic  school  redesigning  is  the  environment  that  supports  and  is 
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complementary  to  the  values,  functions,  and  services  of  new  educational  systems,  This  is  due  to  the 
fact  that,  viewed  as  open  systems,  educational  systems  are  interdependent  with  their  external 
environment. 

State  and  local  policy  support,  which  includes  commitment,  legitimation  and  endorsement, 
alignment,  and  facilitation  should  be  obtained.  Without  policy  support  and  accommodation,  we  can 
hardly  expect  to  implement  envisioned  changes  in  educational  systems  and  instructional 
management  in  the  schools  and  classrooms.  A  broad  base  of  support  and  advocacy  from  the 
business  community,  parents,  and  other  educational  organizations  also  provide  the  support  and 
energy  necessary  for  the  long  journey  to  systems  redesign. 

Commitment  and  Ownership 

Comprehensive  redesign  in  a  system  takes  a  long  time  to  accomplish,  demanding  energy, 
openness,  flexibility,  and  continuous  and  intensive  work  on  the  change  process.  Stakeholders 
participating  in  the  redesign  process  should  consider  themselves  as  'long-distance  runners  (Miles  & 
Ekholm,  1991,  p.  14).  Staff  ownership  and  collaboration  seem  especially  crucial  to  maintaining  the 
momentum  for  change. 

Internal  Structures  and  Culture 

Redesigning  is  more  likely  if  a  school  develops  its  own  internal  capacities  through  the  redesign 
process.  A  school  should  re-conceptualize  peoples'  roles,  relationships,  and  responsibilities.  In 
addition,  it  should  created  an  innovative  culture  that  nurtures  competencies,  morale,  and 
initiatives;  distortion-free  communication  patterns,  shared  decisionmaking,  and  democratic  and 
professional  management  styles. 
Learning/Instructional  System 

A  coherent  learning  and  instructional  system  should  be  designed  based  on  the  reflection  of 
shared  values,  visions,  and  goals.  Without  changes  in  learning  and  instruction,  all  other  changes 
have  little  meaning.  Key  learning  and  instructional  elements  to  be  considered  include  learning 
programs,  learning  strategies,  the  learning  process,  the  learning  environment,  and  assessment. 

Leadership 

Leadership  is  the  most  important  source  of  momentum  in  educational  restructuring  (Blase, 
1986;  Barth,  1990;  Liberman  &  Miller,  1990;  Goodlad,  1983;  Liberman,  1988;  Cohen,  1989).  This 
is  due  to  the  fact  that  purposively  induced  change  processes  require  skillful  and  highly  committed 
management.  Especially,  the  primacy  of  authenticity  and  motivation  in  leadership  have  been 
stressed  in  the  literature. 


2.  Key  Elements:  Learning  from  Case  Studies 

Two  public  schools,  Indiana  Creek  Elementary  School  (ICS)  and  Sunshine  Elementary 
School(SES),  were  selected  through  a  purposive  sampling  strategy  (Kidder,  Judd,  and  Smith,  1986; 
Merriam,  1988),  These  schools  were  researched  at  an  earlier  time  for  different  purposes,  However, 
They  were  selected  for  the  current  study  since  they  had  similarities  as  well  as  differences.  Both 
schools  were  elementary  schools  located  within  a  25  miles  distance  from  one  another  and  were 
designated  as  Indiana  2000  -an  Indiana  state'  restructuring  initiative  project    school  sites.  They 
were  different,  however,  in  terms  of  organizational  and  community  environment.  The  following 
sections  describe  the  methodologies  used  and  the  research  findings  from  each  site. 
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Indiana  Creek  Elementary  School  (ICS) 
A  qualitative  case  study  approach  was  employed  for  four  months  to  gather  data  about  the 
school  setting.  The  methods  of  data  collection  included  participant  observation,  interviews, 
document  analysis,  and  VAXnotes  (an  electronic  conference).  All  the  data  were  recorded  by  means  of 
a  personal  journal 


Data  collection 

The  primary  data  collection  technique  was  observation.  I  visited  and  observed  each  classroom 
once  each  week  from  January  through  April  of  1992.  Whenever  possible,  I  helped  individual 
teachers  as  a  teachers'  assistant  in  the  classroom.  I  also  observed  the  general  atmosphere  of  the 
school,  which  included  students1  activities  outside  the  classroom,  the  classroom  structure,  and  the 
school's  architecture.  Secondary  data  sources  included  interviews  and  document  analysis. 

The  informants  interviewed  included  the  staff,  students,  and  parents.  Interviews  provided 
information  about  informants1  activities,  feelings,  and  lives  (Eisner,  1991),  It  also  was  necessary  to 
analyze  past  events  that  could  not  be  directly  observed  (Merriam,  1988,  p.  72).  The  interviews  with 
the  principal  were  mainly  formal,  conducted  at  her  office,  and  relatively  open-ended.  I  took  notes 
during  the  interviews,  which  were  tape-recorded.  Informal  conversations  were  conducted  with 
teachers,  other  staff,  students,  and  parents  in  the  interviews,  in  classes  or  at  lunch  time. 

Documents  and  programs  also  were  useful  sources  of  information  (Lincoln  &  Guba,  1985)  for 
this  study.  These  were  comprised  of  proposals,  reports  or  articles  prepared  by  the  staff,  the  school 
newspaper,  class  schedules,  the  monthly  school  calendar,  activities  designed  by  the  teachers, 
students'  progress  reports,  and  students'  work,  projects,  and  portfolios. 

Data  Analysis 

I  employed  triangulation  by  using  different  data  gathering  methods  and  different  informants 
or  sources  as  described  above.  Triangulation  allowed  me  to  detect  distortion  and  exaggeration  of 
information  by  checking  the  plausibility  of  the  accounts  and  the  reliability  of  informants  (Whyte, 
1982).  I  also  detected  and  corrected  distortion  by  comparing  an  informant's  account  with  accounts 
given  by  other  informants  (Whyte,  1982).  A  regular  group  meeting  with  other  university  members  in 
the  evaluation  project,  averaging  every  two  weeks  kept  me  probing  possible  biases,  exploring 
meanings,  and  clarifying  interpretations  and  findings  (Schnorr,  1990).  Moreover,  VAXnotes  (an 
electronic  conference)  among  the  evaluation  team*  members  helped  me  to  compare  multiple  points 
of  view  to  detect  and  avoid  bias. 


Geographical  and  demographic  Information  (  ICS  reports.  July.  1991) 

Indiana  Creek  Elementary  School  (ICS)  is  a  K-f>  school  that  serves  over  180  students.  The 
staff  defined  the  socio-economic  level  of  the  local  area  as  low;  fully  one-third  of  the  students  are  on 
the  federal  free/reduced  lunch  program.  In  fact,  over  4U  percent  of  the  adult  population  has  not 
achieved  a  high  school  diploma.  Consequently,  a  large  number  of  students'  potential  was  not 
considered  to  be  met.  The  self-esteem  and  confidence  levels  of  both  parents  and  children  were  very 
low. 

While  ICS  has  only  one  elementary  school  in  its  school  corporation,  the  staff  still  believes  that 
the  elementary  school  can  have  a  dramatic  effect  in  increasing  the  high  school  graduation  rate  of  ICS 
students. 


1  I  worked  with  this  school  as  one  of  the  12  evaluation  team  members.  ICS  invited  us  to 
evaluate  their  redesigning  process. 
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ICS  students,  while  all  Caucasian,  still  represent  a  broad  range  of  cultural  backgrounds. 
While  twenty-five  years  ago  the  area  was  predominantly  an  agricultural  community,  it  is  now 
becoming  a  bedroom  community. 

Process  of  Restructuring 

The  ICS  principal  said,  "The  ideas  for  the  restructuring  of  ICS  were  spawned  at  least  five 
years  ago  during  informal  conversations  of  staff  members  at  lunch,  in  the  hallways,  and  at  other 
gatherings,"  , 

In  the  spring  of  1990,  school  personnel  were  invited  to  participate  in  the  "Restructuring  to 
Promote  Learning  in  America's  School"  teleconference  series  sponsored  by  the  North  Central 
Regional  Education  Laboratory  (NCRKL)  and  the  State  Department  of  Education. 

During  the  summer  of  1990,  the  staff  and  several  parents  attended  conferences,  workshops, 
and  meetings  to  learn  more  about  restructuring, 

As  school  was  beginning  for  the  19904)1  year,  these  individuals  participated  in  the 
redesigning  efforts  unanimously  derided  to  continue  pursuing  their  shared  vision.  The  Design 
Team,  composed  of  a  broad  stakeholder  group  and  outside  facilitators,  sent  surveys  to  the  school 
community.  A  survey  was  conducted  to  determine  what  the  community  wanted  schools  to  be  like  for 
their  children.  Based  upon  the  input  received,  the  Design  Team  developed  a  mission  statement  and 
supporting  principles. 

They  used  this  statement  to  decide  what  issues  they  would  address  first.  Then,  based  on  the 
input  received  from  meetings  open  to  the  community,  they  developed  a  plan  for  their  first  year  of 
redesigning  implementation.  The  major  working  areas,  as  of  the  fall,  of  1992,  were  continuous 
progress,  multi-age  grouping*  alternative  assessment,  thematic  instruction,  cooperative 
learning,  and  integration  of  technology. 

Findings 

Professional  development  was  critical  in  ICS,  which  was  attempting  to  be  a  professional 
development  school  through  collaboration  with  a  higher  education  institution,  active  invitation  of 
outside  help,  research,  and  training. 

"We  will  spend  most  of  the  Indiana  2000  project  grant  money  for  staff  development.  We 
will  use  it  for  teachers  to  visit  other  schools,  to  go  to  conferences,  and  to  invite  workshops 
to  come  into  school  during  the  summer."  (the  principal) 

Teachers  and  the  principal  at  ICS  seemed  to  believe  in  systemic  change  in  which  they  should 
consider  all  possible  elements  simultaneously  and  balance  them  within  the  system.  However,  it 
might  extremely  aggravate  the  uncertainty  and  pressure  on  them. 

"Once  teachers  understood  what  it  was  and  why  we  do  it,  they'd  better  develop  their 

own  skill  themselves."  (the  principal) 

Yet,  she  admitted  that  it  is  almost  too  overwhelming  for  them  and  the  teachers  are 
emotionally  very  exhausted. 

"We  almost  have  to  do  everything  all  at  once,  There's  so  much  to  learn  and  we  don't 
have  time,  Instead  of  focusing  on  portfolio,  alternative  evaluation,  etc.,  I'm  trying  to  put 
them  all  on  the  floor.  That's  because  everything  has  to  go  together." 

is  another  main  element,  and  at  the  same  the  least  available  resource,  in  ICS's  redesigning 
process.  Recently  staff  members  have  cooperated  in  order  to  redesign  curriculum  that  fits  their  own 
needs,  but  have  suffered  from  lfick  of  time. 

"We  don't  have  enough  time  to  talk  with  colleagues  to  share  idens  and  co-work."  (n 
teacher) 


"I  don't  have  time  to  figure  out  what's  going  on  and  whether  any  of  it's  working,"  (a 
teacher) 

Management  of  uncertainty  also  seemed  to  be  a  critical  element  in  ICS's  redesigning 
process.  The  staff  at  ICS  needed  to  manage  outside  complaints  as  well  as  cope  with  business  as 
usual.  However,  they  seemed  not  to  have  a  clear  picture  of  what  they  wanted  to  achieve  after  the 
"disaster,  confusion,  and  grief  involved  in  school  redesign. 

"I  went  to  one  workshop  and  read  some  articles  but  it  is  not  enough  to  help  me.  I  want 
to  see  them  being  used.  I  don't  think  I  have  enough  references."  (a  teacher) 
"...there  is  a  lot  of  enthusiasm,  but  too  many  things  are  going  on  in  class.  The  teachers 
also  worry  about  failure.  They've  heard  about  successful  'individualized'  learning.  But 
one  of  the  teachers  said  that  she  doesn't  know  how  to  apply  it  to  her  class.  And  they 
worry  that  they  can't  handle  that  situation."  (an  outside  evaluator) 

"Wi!  don't  know  the  parameters  of  the  Community  Council  yet  Nobody  can  give  us  the 

answer.  Because  our  school  is  different  from  others,  we  need  the  way  that  fits  our 
school."  (the  principal) 

"  We  do  not  have  a  well-structured  portfolio  yet.  We  still  don't  know  what  kind  of 
portfolio  it  should  be  ,"(a  teacher). 

ICS  did  not  have  much  time  to  demonstrate  success.  In  the  summer  of  1991,  the  School  Board 
gave  the  school  a  one-year  trial  period  in  which  to  implement  changes.  Also,  not  all  of  the  parents 
were  supportive  of  the  school's  efforts.  Some  parents  worried  that  the  changes  would  be 
disadvantageous  for  higher  achievers.  Some  parents  worried  that  the  redesign  efforts  may  not  last 
long. 

"These  teachers  are  very,  very  dedicated.  They  love  kids,  or  they  wouldn't  be  doing  this. 
But  we  all  know  that  people  can  burn  out."(a  parent) 

The  staff  clearly  saw  the  need  for  more  communication  and  collaboration  among  teachers 
and  with  the  community.  They  tried  to  individualize  students'  learning  by  inviting  more  adults  into 
the  classrooms.  Parents  and  community  volunteers  regularly  helped  teachers  inside  and  outside 
each  classroom.  Teachers  were  not  isolated  within  their  classrooms  anymore,  but  opened  them  to 
other  classes  and  the  community.  They  invited  each  other  to  get  involved  in  collaborative  learning 
activities  and  exchange  rewards  and  wishes.  In  this  process,  they  respected  different  perspectives 
and  learning  styles.  Also,  they  cooperated  in  order  to  design  a  curriculum  that  fits  their  needs. 

"The  answer  is  time  and  communication....  What  we  found  is  we  were  using  the 
wrong  kinds  of  communication  with  parents  -written  form.  Parents  in  general  don't  read 
written  documents"  (the  principal). 

Individual  and  organizational  learning  appeared  to  be  a  critical  element  in  their  redesign 
process.  The  staff  and  community  members,  as  individuals  and  as  a  group  tried  to  learp  more 
about  how  to  redesign  their  school  system  in  various  wavs,  A  teacher  referred  to  her  networking 
and  affiliation  with  Phi  Delta  Kappa,  a  state  education  computer  group,  the  school's  design  team, 
the  Illinois  Renewal  Institute,  Technology  2000,  the  21st  Century  Schools  grant,  and  the  3  R's 
grant,  as  well  as  a  small  support  group  of  colleagues  who  were  dedicated  to  the  formation  of  an 
ideal  school. 

Technology,  especially  computers,  was  considered  prominent  in  successful  redesign,  especially 
at  the  level  of  learning  and  instruction.  Computer  activity  was  one  of  the  requirements  in  the  Choice 
programs.  However,  there  was  not  enough  systematic  guidance  in  students'  computer  work,  nor  did 
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the  teachers  seem  to  have  much  preparation  for  teaching  students  to  use  computers. 

"We  would  like  to  utilize  computers  more  and  get  training  for  more  etf:rient  computer 
utilization/'  (teachers) 

Sunshine  Elementary  School  ffiESl 
A  qualitative  case  study  approach  was  used  to  gather  data  from  February  in  1993  through 
August  of  1993,  The  methods  of  data  collection  included  mainly  interviews,  document  analysis,  and 
observation. 

Data  Collection 

Prior  tu  the  first  site-visit,  I  analyzed  available  documents,  including  an  annual  report,  local 
newspaper  dips,  and  some  promotional  pamphlets  which  worn  already  available.  This  provided  me 
with  preparatory  information  about  the  Hite.  Upon  visiting  the  school,  I  discussed  my  research  plan 
with  the  principal,  At  this  pre- research  meeting,  I  obtained  more  documents,  which  included  an 
Indiana  12000  application  and  I'HA(l'erfnrmance  Based  Accreditation)  documents. 

The  primary  data  collection  technique  involved  interviews  with  the  stakeholders  most  involved 
in  the  redefining  of  their  school  system.  These  included  an  administrator,  two  teachers,  t"'o 
parents,  and  one  community  member.  I  mainly  conducted  semi-structured  interviews.  These 
interviews  were  guided  by  a  list  of  questions  or  issues  to  be  explored,  hut  neither  the  exact  wording 
nor  the  original  order  of  the  questions  was  used  during  the  interviews.  In  addition,  structured 
interviews  were  added  to  obtain  some  standardised  information  ( Merriam,  1988,  p.  74). 
Information  was  collected  about  the  past,  present  and  future  by  asking:  What  has  the  school  done, 
how,  and  why?  What  have  they  been  doing,  how,  and  why?  What  are  they  doing,  how,  and  why? 

Key  informants  were  interviewed  individually  at  least  twice.  Follow-up  interviews  were  made 
either  through  a  meeting  or  phone  call.  I  took  notes  during  the  interviews,  which  were  audio  tape- 
recorded.  Verbatim  transcriptions  were  made  from  the  recorded  interviews.  As  a  non-participant 
observer,  I  observed  classroom  activities.  After  I  left  the  room,  I  summarized  my  observations  as 
soon  as  possible  eit>  er  on-  or  off-site.  Then  I  took  time  to  remember  more  substance  and  elaborated 
the  scenes  and  dyna  Tries  at  the  site.  Data  from  documents  and  observations  were  recorded  for  the 
purpose  of  verification. 

Data  Analysis 

The  techniques  '»  "unitizing'  and  'vatvgorizintf  (Lincoln  &  (tuba,  1985,  p.  344)  were  used  to 
analyze  data.  All  data  from  interview  transcripts,  documents,  and  observation  notes  were  read 
through  several  time*  \  rom  beginning  to  end.  While  reading,  1  jotted  down  notes,  comments,  and 
observations  in  the  margins  (Ooetz  and  lA'Comple,  l!)H4,  p.  11)0- 11)1).  Through  this  process,  I 
identified  units  of  infoi  nation  as  the  basis  for  defining  categories.  Then,  selected  information  was 
summarized  and  categorized  through  the  use  of  cuc/c.s. 

Developing  eateg«  >ries  involves  seeking  recurring  tXUldOHlkli  in  flata  by  comparing  one  unit  of 
information  with  the  n  <t.  Once  the  relevant  sets  of  categories  are  derived  from  the  data,  they  can 
he  filled  oot  and  made  i  lore  vigorous  by  searching  through  the  data  for  more  and  better  units  of 
information.  An  alternative  display,  a  matrix,  was  utilized  for  hotter  analysis. 

Interview  t\ntn  auaJyjUfl,  The  data  consisted  of  verbal  accounts,  derived  from  JM  hours  of 
interview  interactions.  Upon  completion  of  all  interviews,  the  contact  summaries  were  edited  and 
sent  to  informants.  Kach  informant  was  called  after  they  received  the  summarized  interviews  - 
interview  summary  reports  --for  initial  verification  of  content.  Minor  editing  was  done  through  phone 
calls  or  personal  contact**,  according  to  participants'  preferences.  Upon  conducting  these  verification 
contacts,  there  was  one  follow-up  interview. 

Document  Analysis.  Documents  also  were  summarized  and  coded  on  the  summary  sheet. 
Primarily,  the  content  analysis  focused  on  measuring  the  frequency  and  variety  of  information, 
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confirming  the  emergent  hypotheses,  and  advancing  new  categories  and  hypotheses  (Mr  rriam,  1988, 
p.  108). 

Observation  analysis.  Data  collected  through  observation  was  summarized  and  coded  on 
observation  summary  sheets.  The  main  focus  in  the  observation  was  to  answer  the  question,  "Are 
they  really  practicing  what  they  said  they  value?"  The  main  purpose  of  the  observation  was  for 
trianeulation. 


Demographics  of  gfihflfll  building 

Sunshine  Elementary  School  (SES)  was  a  relatively  large  elementary  school  of  530  students. 
The  faculty  describe  their  school  as  being  somewhere  between  rural  and  suburban.  However,  they 
think  they  are  close  to  being  a  suburban  school. 

There  were  drastic  changes  and  growth  in  the  school  community  between  1989  and  1991  with 
the  construction  of  a  large  new  school  building  in  1990.  50%  of  the  families  and  75%  of  the  faculty 
are  new.  In  1989,  the  school  had  11  staff  members  but  by  1992  it  had  52.  Most  parents  were 
employed  in  light  manufacturing  or  service  sector  jobs.  Four  to  five  percent  of  the  students  were 
ethnic  minorities,  and  some  used  English  as  a  second  language.  One  of  the  faculty  members  said 
that  most  of  the  families  were  non-conventional,  that  is  two-parent  and  two-income,  or  single  parent 
and  single  income  families,  10%  of  the  school  population  was  enrolled  in  the  before-  and  after-school 
custodial  program. 

Process  of  Restructuring 

The  current  restructuring  effort  was  initiated  by  the  faculty,  who  saw  that  what  they  were 
doing  was  not  successful  and  thus  wanted  to  make  changes.  "Many  of  the  faculty  felt  that  much  of 
the  business  we  were  doing  did  not  meet  the  needs  of  the  kids.  We  were  seeing  that  many  kids 
were  not  successful  with  what  we  were  doing  (the  principal).  "The  driving  forces  are  collaborative 
staff1  (a  teacher),  "having  leadership  skills  and  professional  excellence,  having  broader  pictures. .  . 
sharing  knowledge."  (a  teacher). 

The  starting  point  of  the  current  restructuring  effort  was  differently  defined  by  each  informant. 
It  was  clear,  however,  that  they  began  to  think  about  and  prepare  for  redesigning  their  school  before 
the  new  building  was  open.  They  have  been  involved  in  the  National  Reading  Styles  Institute  as  a 
model  school  site  since  1988. 

In  1989,  they  doubled  staff  size  and  took  time  to  get  to  know  each  other's  expertise.  "It  was 
an  active  reorganization  and  data  collection  time  for  us."  (a  teacher). 

In  1990,  a  new  building  was  opened  and  a  PBA  (Performance-Based  Accreditation)  document 
was  produced  through  the  group  work  of  faculty  members,  which  consisted  of  background 
information  on  the  school  community,  parent  survey  results,  a  mission  and  belief  statement,  goals 
for  SES  ,  analysis  of  strengths  and  weaknesses,  and  goals  and  strategies  to  improve  these  areas. 

In  1991,  SES  began  an  ungraded  primary  multi-age  grouping  program,  UP-LIFT(Ungraded 
Primary  -  Learning  Is  Fun  together).  In  the  same  year,  they  started  to  talk  about  a  strategic  way  of 
looking  at  outcome-based  instruction. 

In  1992,  they  clarified  'Exit  Outcome'  which  is  what  they  expect  students  to  know  and  to  do 
when  they  leave  school.  In  1992-93,  they  expanded  multi-ungraded  grouping  by  starting  an 
intermediate  multi-age  grouping,  called  Project  LaUnched.  At  the  end  of  1992,  they  developed 
outcome-based  instruction  as  a  faculty. 

In  1993,  they  tried  to  more  fully  implement  OBE(Outcome-based  education),  with  each  faculty 
member  being  required  to  develop  his/her  own  education  plan.  They  currently  put  a  lot  of  effort  into 
making  connections  among  various  groups  within  the  building. 
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Findings 

Continuous  and  focused  professional  development  was  one  of  the  most  commonly  discussed 
elements  among  informants,  which  was  defined  in  terms  of  "collaboration"  and  "collegiality"  among 
staff,  as  well  as  "self-efficacy."  ■ 

"In  order  to  achieve  the  learning  consistent  with  our  beliefs  support  from  people,  and 

staff  development  is  really  important."  (a  teachers) 

"at  SES,  the  successful  factors  for  restructuring  is  collaborative  staff,  people  who  are 
collegia!  and  are  not  threatened  by  the  opportunity."  (a  teacher) 

Collective  wisdom  emerged  as  another  key  shared  element  among  the  informants.  They 
believed  that  they  needed  to  listen  to  new  and  different  ideas  and  perspectives  from  more  people 
both  within  and  outside  the  building. 

"I  believe  the  community  should  be  involved  in  their  [SES'SJ  reHtructuring  process  for 
fresh  perspectives.  Homebody  outside  the  system  has  some  new  and  fresh  ideas,  not 
always  the  same  old  people  are  talking  to  the  same  old  people  about  the  same  old  thing. 
It  is  valuable  to  bring  in  new  ideas  because  new  ideas  are  the  stimulus  for  change  and 
there's  also  a  result  of  change.  That  is  new  ideas  to  keep  u  process  evolving  so  it  wont' 
be  stagnant,  (a  community  member) 

Administrative  leadership  also  was  discussed  as  one  of  the  primary  elements  in 
restructuring:  "Leadership  is  critical."  Administrative  leadership  at  SES  was  perceived  as  a 
successful  factor  in  their  redesigning  process. 

"At  SES,  the  principal  allows  to  be  collegia!  and  is  collegial  himself.  He  does  not  see  me 
inferior  and  I  do  not  see  him  superior.  We  see  that  each  other  has  specific  roles  and  we 
have  expectations  for  each  others  for  our  role."  (a  teacher) 

"The  approach  I've  used  in  terms  of  bringing  people  along  is  collaboration  and 
encouragement,  and  direction  without  dictation."  (the  principal) 

Parent  involvement  in  the  redesigning  process  appeared  to  be  critical.  Parents  themselves 
were  convinced  about  the  importance  of  their  support  and  involvement  more  than  any  other 
informants. 

"If  you're  not  willing  to  come  and  talk  at  the  meeting  and  express  your  opinion,  you 
don  t  have  a  right  to  complain."  (a  parent) 

The  staff  members  also  believed  that  parent  involvement  was  one  of  the  'should-bc'  factors  in 
successful  redesigning: 

"Parents  should  be  part  of  the  change  process  from  the  beginning.  .  .  .  They  are  not 
necessarily  academic  educators,  but  they  should  be  understanding  and  motivated  as 
parents  so  that  they  should  definitely  be  a  major  part  of  the  change  process."  (a  teacher) 

Community  involvement  As  indicated  by  their  documents,  the  people  at  SES  were  mainly 
interested  in  'community'  as  'resources'  which  allowed  them  to  have  'a  school-wide  planning  time.1 
In  the  community  member's  part,  community  also  perceived  as  an  outside  helper  of  SES's 
restructuring  process. 

"The  other  element  of  my  role  would  have  been  to  create  a  climate  for  acceptance  of 
change  within  the  overall  school  administration.  So  it  would  be  a  kind  of  paving  the  way 
for  the  principal  at  the  upper  level  and  further  try  to  reinforce  what  he  wa*  doing"  (a 
community  member). 
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Communication.  No  matter  what  or  who  is  involved  in  the  change  process,  communication 
appears  to  be  a  critical  factor.  Streamlined  communication  within  the  school  was  emphasized  as 
"the  heart  of  what  has  to  happen  whenever  people  have  to  work  together  with  understanding11  (the 
principal). 

"Just  sharing  ideas  can  charge  people  up.  If  not,  we  can  be  very  isolated.  Through 
communication  with  others  we  can  take  the  best  ideas  and  use  them.  Also  you  can  iron 
things  out,  like  misunderstandings."  (a  teacher) 

Choice.  Choice  appeared  to  be  almost  a  norm  at  SES.  Staff  and  parents  understood  that 
changing  can  be  uncomfortable  for  many  people  and  thus  people  should  have  the  final  say  regarding 
their  own  practice.  By  practicing  the  change  strategy  of  'school  within  a  school',  they  attempted  to 
"shape  people's  perception,  not  force  them  to  be  a  part  of  change"  (the  principal).  They  also  believed 
that  change  can  not  happen  overnight. 

"You  don't  make  anybody  do  it.  It  is  always  voluntary  hero,  Then  people  who  feel 
uncomfortable  with  change  will  have  a  place  to  he  rather  than  junt  complain."  (u  teacher) 

Ownership.  "Ownership"  of  stakeholders  was  assumed  to  he  an  element  fostering  the 
restructuring  process.  The  principal  defined  their  restructuring  process  as  a  'bottom-up'  approach 
where  the  impetus  for  change  comes  from  the  'grassroots',  not  from  the  administration. 

"We  worked  as  a  whole  in  order  to  discuss  what  we  stand  for  and  what  we  believe  about 
kids....  The  model  works  because  of  its  belief  driven  and  self-driven  nature.  It  is  a 
grassroots  kind  of  thing  so  it  works."  (a  teacher) 

"SES's  approach  toward  the  community  is  very  good.  If  you  need  community  help  you 
need  to  bring  the  community  into  it  at  the  early  stages.  You  can't  just  make  a  change 
and  then  ask  everybody  to  support  your  change.  I  think  THE  PRINCIPAL  understood 
that  and  involved  community  from  the  first.  So  people  are  part  of  the  change,  not  just 
recipients."  (a  community  member) 

Resources  were  prominent,  especially  in  the  staff  members'  discourse  as  well  in  their  main 
documents.  The  informants'  assertions  about  their  accomplishments  provided  good  examples  of  the 
most  appreciated  resources:  legitimacy  of  change,  money,  and  networking  with  other 
organizations. 

"The  designation  as  an  Indiana  2000  school  site  was  significantly  important  since  it 
legitimized  what  we  were  doing  throughout  the  state  and  the  school  district.  Also,  those 
teachers  that  participated  in  the  development  of  the  grants  had  a  chance  to  be 
recognized  and  increase  their  feeling  of  self-efficacy.  It  also  gave  us  monetary  resources 
that  we  primarily  spent  on  staff  development"  (the  principal). 

"Designation  as  Indiana  2000  gave  us  a  step  of  approval  from  the  state.  So  politically 
good.   And  it  gave  us  money  so  we  could  do  staff  development.   It  also  brought  us 
network  with  other  people  in  Indiana  2000.  Through  the  network,  we  can  support  each 
other.  I  think  recognition  is  the  main  thing"  (a  teacher). 
SES'  belief  about  networking  was  well  exhibited  through  its  'outreach'  effort.  The  school 
hosted  its  own  'professional  development  conference'  in  which  it  invited  people  to  see  specific 
elements  of  what  was  being  done  at  SES,  what  they  thought  was  unique  about  SES,  and  perhaps 
unique  for  their  county. 

"We  wanted  to  take  pride  in  what  we  are  doing  and  communicate  it  with  other  learning 
environments.  We  believe  what  we're  doing  run  be  transferred  to  other  environments." 
(a  teacher) 
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Time  was  another  resource  that  is  considered  to  be  a  critical  factor  in  redesigning,  yet  not 
provided  enough: 

"If  we  are  going  to  change  the  ways  we  do  the  business,  we  need  to  have  time  to 
collaborate"  (the  principal),  "communicate"  and  "coordinate"  a  school-wide  planning 
(PBA);  "during  the  school  day."  (MM) 


Conclusion  and  discussion 

The  literature  review  allowed  for  considering  the  key  elements  to  success  in  comprehensive 
ways,  revealing  internal  and  external  factors  as  well  as  human,  structural,  cultural,  and  procedural 
factors.  The  findings  of  the  case  study  mainly  confirmed  or  elaborated  on  existing  theory. 

From  these  two  research  approaches;  literature  review  and  the  case  study,  four  elements  were 
identified  ns  being  the  most  critical  for  a  successful  redesigning  process:  resources,  change 
management,  an  appropriate  internal  structure  and  culture,  support  from  the 
environment  and  clearly  shared  visions.  Yet,  we  should  understand  that  those  elements  work 
within  the  context  of  the  change  process  and  that  other  elements  might  be  more  critical  in  different 
cases. 

Interestingly,  however,  the  findings  from  the  case  study  of  the  two  research  sites  did  not 
provide  information  indicating  the  schools  held  a  'clearly  shared  vision*  was  a  critical  element 
enhancing  their  redesigning  process,  unlike  tV  •  literature  review.  Thus,  it  seems  important  to 
answer  the  question,  what  does  this  differen  vi  between  other  researchers'  conclusions  and  my  own 
results  mean? 

Since  change  is  assured  by  "a  favorable  configuration  of  key  aspects"  (Miles  and  Ekholm, 
1991),  when  certain  elements  are  seriously  considered,  other  elements  also  should  be  emphasized  in 
order  to  produce  the  effect  as  planned.  In  addition,  different  schools  or  school  districts  might  have 
their  own  conditions.  Then,  different  sites  need  to  consider  different  elements  in  order  to  respond  to 
their  own  need  in  change  process.  As  redesign  emerges,  key  elements  to  be  considered  may  shift 
since  different  design  phases  might  demand  different  elements.   But  this  is  not  fixed,  being  a 
eontextunl  and  relative  matter  depending  on  what  happened  before,  what  has  been  going  on 
recently,  and  what  should  happen  in  the  course  of  redesign  process. 
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Over  40%  of  freshmen  entering  public  technical  schools,  colleges  and  universities  are 
seriously  deficient  in  reading,  English  skills,  and  mathematics  skills.  Many  students  are 
unable  to  improve  and  eventually  drop-out  of  college,  despite  remedial  efforts.  This  study 
used  closed -captioned  videotape  as  a  technological  approach  to  improving  learning  and 
retention  skills  in  college  students  with  deficiencies  in  reading.  An  experimental  group  of 
108  Developmental  Studies  College  students,  along  with  168  comparison  academic  core 
education  students,  were  exposed  to  one  of  six  treatments  (i.e.,  closed-captioned  video  with 
sound,  regular  video  with  no  captioning,  closed-captioned  video  with  no  sound,  audio  tape 
with  print,  audio  tape  without  print,  and  print  only).  Data  gathered  were  pretest,  postt  -U, 
and  retention  test  scores. 

Even  though  gains  were  made  in  posttest  and  retention  test  scores  for 
Developmental  Studies  students  in  all  treatments,  they  never  achieved  to  the  level  of  thinr 
academic  peers.  When  examining  individual  treatments,  the  use  of  closed-captioned  video 
was  significantly  better  than  other  treatments  for  academic  core  students;  however,  for 
Developmental  Studies  students,  this  treatment  produced  the  poorest  scores  for  the  posttest 
and  second  poorest  for  the  retention  test. 

This  study  suggests  learning  through  multi -sensory  instructional  media,  such  as 
that  commonly  employed  in  academic  core  college  classrooms,  is  more  difficult  for  the 
Developmental  Studies  student. 


A  substantial  portion  of  college  students'  academic  competency  is  reflected  in  reading 
and  English  skills.  Deficits  in  reading  and  English  usage  result  in  poor  academic 
achievement,  which  leads  to  academic  probation  and  subsequent  withdrawal.  There  are  no 
%vinners  when  a  college  student  struggles  in  reading  and  writing.  Students  who  drop  out  of 
college  because  of  reading  and  writing  deficits  will  find  employment  in  lower-paying  positions 
(Johnson,  1987;  Kerachsky  &  Thornton,  1987;  Rusch  &  Phelps,  1987). 

Developmental  studies  and  student  support  programs  are  designed  to  provide 
academic  support  to  students  who  are  underprepared  in  the  basic  fundamentals  of  reading, 
writing,  mathematics,  and  study  skills.  Many  enter  college  lacking  appropriate  secondary 
school  coursework,  but  some  of  these  students  have  disabilities.  The  majority  of  post- 
secondary  students  with  a  disability  are  learning  disabled;  however,  many  never  inform  the 
college  of  their  disability,  fearing  a  stigma  will  be  placed  on  them  (Henderson,  1992). 
Additionally,  college  programs  have  no  standard  definition  of  learning  disabilities  (Wallace 
&  McLoughlin,  1988).Thus,  it  is  common  for  students  to  be  inappropriately  served, 
underserved,  or  to  receive  no  support  services  at  all. 

Despite  efforts  by  most  colleges  and  universities,  traditional  student  support 
programs  that  require  specialized  classes  are  not  highly  successful.  Numbers  of  students 
"exiting"  these  programs  can  be  misleading,  for  many  simply  drop  out  of  college  (Whinnery, 
1992).  There  are  indications  numbers  of  students  needing  remediation  are  increasing. 
Since  the  end  of  1970,  the  percent  of  freshmen  who  report  having  a  disability  has  tripled. 
Additionally,  the  disabling  conditions  that  are  most  prevalent  today  are  more  likely  to  he 
invisible  (e.g.,  learning  disabilities,  health  impairments,  communication  deficits,  low  vision, 
or  hearing  impairments)  than  obvious  disabilities  (e.g.,  deafness,  blindness,  physical) 
(Henderson, 1992)  .   New  provisions  for  appropriate  student  access  and  support  under  \\w 
1990  Americans  with  Disabilities  Act  challenge  colleges  and  universities  to  examine  more 
efficient  and  effective  ways  to  deliver  appropriate  learning  environments,  methods,  and 
materials  to  students  who  desire  a  higher  education. 

Reading  skills  for  college  students  are  essential  forsuceess.  Knglish  and  reading 
programs  should:  (a)  emphasize  critical  thinking,  (b)  build  students'  elaboration  skills,  (e) 
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advance  readers'  abilities  to  interpret,  and  (d)  ufford  opportunities  to  form  opinions  with 
reasoned  support  (Applebee,  Langer,  &  Mullis,  1988).  In  order  to  achieve  these  goals, 
students  must  develop  fluency  in  reading  as  well  as  comprehension  skills.  Carver  and  Darby 
(1972)  believe  these  two  skills  are  inseparable.  They  define  reading  comprehension  as: 

...a  thorough  communication  process  which  involves  two  primary  components 

■  the  rate  at  which  the  thoughts  are  received  and  the  accuracy  with  which 

the  thoughts  arc  understood.  The  end  product  of  these  two  components  is  the 

efficiency  with  which  the  thoughts  are  communicated  (p. 292). 

Poor  reading  comprehension  is  usually  the  first  academic  identifier  that  a  student 
may  have  a  specific  learning  disability.  (Smith  &  Luckasson,  1992,  p.  217).  For  these 
students  who  stay  in  school  until  their  senior  year,  their  average  academic  achievement  in 
reading  is  at  the  fifth  grade  level  (Schumaker,  Deshler,  Alley,  &  Warner,  1983).  Therefore, 
students  who  have  learning  disabilities  are  reading  at  approximately  this  levol  when  they 
enter  college.  To  ensure  success  at  the  college  level,  new  methods  and  materials  must  be 
developed  to  address  learner  diversity  through  media,  such  as  instructional  video 
(Hofmeister,  1992).  Closed-captioned  educational  videotape  used  to  instruct  college 
students  who  are  underprcpared  or  have  disabilities  may  prove  to  he  a  powerful  alternative 
or  addition  to  traditional  support  methods. 

Much  of  the  current  research  with  closed-captioned  video  has  been  done  with  bearing 
impaired  populations  (Koskinen,  Wilson,  &  Jensema,  1985;  Montandon,  1982;  Sherman  & 
Sherman,  1989)  and  with  persons  who  use  English  as  a  Second  Language  (ESL)  (Markham, 
1989;  Spanos  &  Smith,  1990).  Results  are  encouraging  that  closed-captioning  is  effective  in 
enhancing  learning  and  improving  literacy  skills. 

The  majority  of  reading  studies  that  examined  closed-captioned  materials  was 
conducted  with  students  in  elementary  grades  (Brashier-Spath,  1989;  Koskinen,  Wilson, 
(Jambrell,  &  Neuman,  1993;  Beentjes  &  Van  Der  Voort,  1988;  Goldman  &  Goldman,  1988). 
Improvement  in  vocabulary,  reading  comprehension,  writing  skills,  and  reading  rates  was 
reported.  Based  on  these  studies,  effects  of  closed-captioned  materials  for  reading  deficient 
post-secondary  students  are  examined.  Can  post-secondary  students  who  have  intense 
reading  task  demands  do  better  using  closed-captioned  video  media? 

The  purpose  of  this  study  was  to  examine  the  effects  of  closed-captioned  educational 
video  as  an  instructional  tool  to  improve  reading  comprehension  and  retention  of  learning. 

METHOD 

Subjects 

This  study  included  168  students  enrolled  in  Developmental  Studies  curriculum 
(English  and  Heading  classes)  and  168  students  from  Regular  academic  core  courses  (School 
of  Education  classes).  All  subjects  were  first  year  students  at  a  regional  university  in  the 
Southeast, 

Subject  selection  for  the  Developmental  Studies  students  was  based  on  criteria  from 
the  Freshmen  Evaluation  for  Developmental  Studies,  an  evaluation  tool  comprised  of  writing 
samples  and  test  scores.  Students  scoring  below  400  on  the  Scholastic  Aptitude  Test  (SAT) 
Verbal  and  below  75  on  the  Collegiate  Placement  Exam  reading  and  English  subtests  were 
included  in  the  subject  pool.  All  students  who  participated  in  the  study  volunteered, 

The  Regular  academic  core  course  students  met  college  admission  standards  and 
had  SAT  Verbal  scores  at  400  or  above  and  scored  75  or  above  on  the  Collegiate  Placement 
Exam  in  reading  and  English  subtests. 

There  were  significantly  more  African-American  Developmental  Studies  students 
than  Regular  academic  core  students  who  volunteered  for  the  study.  Additionally,  there 
were  significantly  more  females  in  the  Regular  academic  core  group  than  males.  These 
differences  also  reflect  numbers  of  students  in  the  subject  pool. 
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Insert  Table  1:  Demographic  Data  here 


Materials 

A  thirty  minute  edited  closed-captioned  video  from  The  Civil  War:  Episode  One,  The 
Cause  —  1861  (Burns,  1990)  was  the  treatment  material.  Narration  of  the  video  was 
recorded  on  audio  cassette  as  well  as  transcribed  in  printed  form.  Materials  needed  to 
administer  treatments  were:  2  TeleCaption  4000  closed-captioned  decoders,  2  audio 
cassette  players,  3  Panasonic  BT-S1300N  color  video  monitors,  and  3  Sharp  XA-305  video 
cassette  players.  Text  was  double  spaced  and  administered  in  bound  booklets. 

Measures 

Based  on  the  edited  video,  a  33 -item  multiple  choice  test  was  constructed  and  used 
as  a  pretest,  posttest,  and  retention  test.  The  items  tested  subjects'  ability  in  decoding 
single  words,  understanding  vocabulary,  interpretation  of  sentences  (including  appreciation 
of  morphology  and  syntax),  identifying  main  ideas,  identifying  supporting  details,  rejecting 
irrelevant  or  distracting  information,  retelling  a  passage,  identifying  the  author's  intention 
and/or  point  of  view,  and  summarizing.  These  are  considered  relevant  parameters  of  reading 
comprehension  necessary  for  reading  success  (Levine,  1987,  p.  298). 

Experimental  Design 

The  experiment  was  a  2  x  6  x  3  (Group  x  Treatment  x  Test  Scores)  within  factor 
design  using  treatment  and  test  scores  as  factors.  Two  groups  were  Developmental  Studies 
students  and  Regular  academic  core  curriculum  students.  Three  tests  were  pretest,  posttest, 
and  retention  test.  Six  treatments  and  their  descriptions  were: 

C  C      Closed-Captioned  Video. 

Students  receiving  this  treatment  viewed  a  video  with  sound  having  closed-captioned 
text  shown  at  the  bottom  of  the  video  monitor. 
V         Sound  Video. 

Students  receiving  this  treatment  viewed  a  regular  video  with  no  captioning. 
CM      Muted  Closed-Captioned  Video. 

Students  viewed  a  video  with  captioned  text  but  the  audio  portion  was  muted.  The 
purpose  of  this  treatment  was  to  test  subjects'  visual  sensory  adapting  capability 
when  the  situation  required  them  to  process  complex  information  (video  and 
captioned  text)  when  audio  sensory  mode  was  artificially  disabled. 
A        Audio  Media. 

Video  narration  was  recorded  on  an  audio  cassette  tape  format. 
AP      Audio  and  Printed  Media. 

Students  receiving  this  treatment  listened  to  audio  tape  narration  as  well  as  read 
the  printed  narration  at  the  same  time. 
P         Printed  Media. 

Students  read  a  printed  text  transcribed  from  the  video  narration.  (This  treatment 
served  as  control.) 

Based  on  the  grouping  and  treatment  assignment  described  above,  a  grouping  chart 
is  depicted  as  follows: 


Insert  Figure  1:  Grouping  Chart  here 


This  study  examined  the  following  questions: 

1 .         Between  groups,  do  Developmental  Studies  students  perform  differently  from 
Regular  ac  ademic  core  curriculum  students  in  their  mean  pretest  scores,  mean 
posttest  scores,  and  mean  retention  test  scores?  (OS  vs  RA) 
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2.  Between  groups,  do  Developmental  Studies  students  and  Regular  academic 
core  curriculum  students  receiving  the  same  treatment  perform  differently  in  their 
mean  pretest  scores,  posttest  scores,  and  retention  test  scores?  (1  vs  7,  2  vs  8,  8  vs 
9.  4  vs  10,  5  vs  11,  and  6  vs  12) 

3.  Within  groups,  are  there  differences  among  groups  of  subjects  who  received 
different  treatments  in  their  mean  pretest  scores,  mean  posttest  scores,  and  mean 
retention  test  scores?  (1  vs  2  vs  3  vs  4  vs  5  vs  6,  and  7  vs  8  vs  9  vs  10  vs  11  vs  12). 

4 .  Within  groups,  are  there  differences  among  groups  of  Hubjects  who  received 
closed-captioned  video,  regular  video,  and  printed  media  in  their  mean  pretest 
scores,  mean  posttest  scores,  and  mean  retention  test  scores?  (1  vs  2  vs  3,  and  7  vs 
8vs9) 

5.  Within  groups,  are  there  differences  among  subjects  who  received  the  closed- 
captioned  with  sound  video  treatment  from  subjects  who  viewed  the  sound  video 
treatment  in  their  mean  pretest  scores,  mean  posttest  scores,  and  mean  retention 
test  scores?  (1  vs  2,  and  7  vs  8). 

Procedures 

All  subjects  were  given  a  33-item  pretest  on  the  topic  The  Civil  War:  Episode  One,  The 
Cause  —  1861  (Bums,  1990)  containing  information  presented  in  the  treatment  session.  Those 
scoring  80%  correct  and  higher  on  the  test  were  eliminated  from  the  study  because  they  had 
mastered  the  instructional  materials.  After  the  subject  pool  was  identified,  a  Pilot  Study  was 
conducted  with  36  students  to  correct  any  errors  and  modify  treatment  materials. 

In  the  Experimental  Study  volunteer  students  in  Developmental  Studies  curriculum 
and  students  in  Regular  academic  core  courses  were  randomly  assigned  to  one  of  the  six 
types  of  treatment  listed  above.  For  both  groups,  each  cell  contained  25  students. 
Immediately  after  treatment,  subjects  were  given  a  posttest  with  the  same  33-item 
multiple  choice  test  items  used  in  the  pretest.  Four  weeks  after  the  posttest,  the  subjects 
were  given  the  same  33-item  multiple  choice  test  as  a  retention  measure.  All  data  was 
analyzed  and  interpreted  to  answer  the  experimental  questions  stated  above. 

RESULTS 

The  dependent  variables  for  the  study  are  mean  pretest  scores,  mean  posttest 
scores,  and  mean  retention  test  scores.  Test  items  of  the  pretest,  posttest  and  retention  test 
were  constructed  in  a  multiple  choice  format. 

Rased  on  the  experimental  questions  stated  in  Design  section  under  Method,  all 
data  were  analyzed  using  the  following  statistical  analysis  procedures  with  a  significance 
level  set  at  a  =  .05.  Statistical  analyses  of  all  data  in  this  investigation  were  performed 
through  the  use  of  SAS  statistical  software. 

Question  1:  Multivariate  analysis  of  variances  (MANOVA)  was  used  to  test  group 
differences. 

Question  2:  Profile  analysis  was  conducted  to  test  group  by  treatment  interaction  effect.  If 
significant  differences  were  found,  treatment  analysis  was  performed  using  oneway  ANOVA 
and  contrast  analysis. 

Question  3:  Profile  analysis  was  conducted.  If  significant  differences  were  found,  a  oneway 
ANOVA  for  each  group  was  performed,  with  post  hoc  comparisons  of  means  via  the  Tukey-B 

tost. 

Orthogonal  contrast  analyses  were  conducted  on  both  Question  4  and  Question  5. 

Analysis  of  Covariance  (ANCOVA)  procedures  were  conducted  on  the  after-treatment 
measures  of  mean  posttest  scores  and  mean  retention  test  scores.  The  covariate  for  the 
mean  posttest  scores  was  the  mean  pretest  scores.  Both  mean  pretest  scores  and  mean 
posttest  scores  were  used  as  covariates  for  the  analysis  of  mean  retention  test  scores 
differences. 
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Mean  scores  of  students'  performances  on  pretest,  posttest,  and  retention  tests  are 
listed  in  Table  2. 


Insert  Table  2:  Means  Table  here 


Question  #1 

A  multivariate  analysis  was  performed.  Group  differences  were  found  on  all 
measures  (i.e.,  pretest,  posttest,  retention  test).  Developmental  Studies  students  scored 
statistically  below  their  Regular  Enrolled  peers  before  and  after  treatment.  See  Figure  2. 

MANOVA  for  Question  1 
Dependent  Variable:  Pretest. 


Source 

DF 

SS 

MS 

F 

P 

GROUP 

1 

1172.16 

1172.16 

87.08 

<001 

ERROR 

298 

4011.23 

13.46 

TOTAL 

299 

5183.40 

Dependent  Variable: 

Posttest. 

Source 

DF 

SS 

MS 

F 

p 

GROUP 

1 

2670.08 

2670.08 

132.91 

<001 

ERROR 

298 

5986.75 

20.09 

TOTAL 

299 

8656.84 

Dependent  Variable: 

Retention  Test 

Source 

DF 

SS 

MS 

F 

p 

GROUP 

1 

2144.01 

2144.01 

119.90 

<001 

ERROR 

298 

5328.93 

17.88 

TOTAL 

299 

7472.95 

Insert  Figure  2:  Group  Comparison  of  Three  Measures  here 


Question  #2 

The  following  measures  were  performed:  For  the  pretest  measure,  group  by 
treatment  ANOVA;  posttest,  group  by  treatment  ANCOVA  using  pretest  as  the  covariate; 
retention  test,  group  by  treatment  ANCOVA  using  pretest  and  posttest  scores  as  covariates. 

The  results  of  the  analyses  on  the  three  measurements  show  group  differences  but 
no  treatment  or  group  by  treatment  interaction  effects.  Developmental  Studies  students* 
scores  were  significantly  lower  than  Regular  Enrolled  students  scores  for  all  measures. 

Question  #3 

(For  Questions  #3,  #4,  #5,  the  analyses  were  performed  to  determine  treatment 
effects  within  groups.) 

The  following  measures  were  performed:  For  the  pretest  measure,  one-way  ANOVA; 
posttest,  one-way  ANCOVA  using  pretest  as  the  covariate;  retention  test,  one-way  ANCOVA 
using  pretest  and  posttest  as  covariates. 

For  Developmental  Studies  students,  no  treatment  differences  were  found  in  pretest 
and  posttest  scores.  There  was  a  significant  difference  in  retention  test  scores.  Tukey's 
Studontized  Range  Test  indicated  students  who  received  muted  video  and  those  who 
received  printed  media  scored  better  than  those  who  received  closed -captioned  video  and 
audio  only.  See  Figure  3. 


Insert  Figure  3:  Selected  Treatment  Comparison  of  Developmental  Studies  Students' 

Measures  here 


For  Regular  Enrolled  students,  no  treatment  differences  were  found  in  pretewt  Heores. 
There  was  a  significant  difference  in  posttest  scores.  Tukey's  Studentized  Range  Tent 
indicated  students  who  received  audio  print  scored  higher  than  those  students  who  received 
print  only  and  audio  only  treatments.  There  was  a  significant  difference  in  retention  sroroM. 
Tu key's  Studentized  Range  Test  indicated  students  who  received  audio  print  find  Honed 
captioned  treatments  scored  significantly  higher  than  those  students  who  received  print  onlv 
treatment. 

ANOVAs  &  ANCOVAs  for  Question  3 
Dependent  Variable:  Retention  Test  (Developmental  Studies  StudentH). 
Source  DF  SS  MS  V  p 

TREAT  5  178.37  35.67  fi.oV        <-  001 

Pretest  1  408.29  408.29        03.72        *  (Nil 

Posttest  1  fi20.9f>  fi20.9fi        Ufi  111        '  001 

FUROR  142  BOU.HH  B.4I 

TOTAL  149  21 17.40 

Dependent  Vannble*  I 'owl  tent  (Regular  Enrolled  Students). 

Source                        DF                    SS                 MS  F  p 

TREAT                        ft                231  00             40.32  II  07        '  01 

Pretest                          1               I20o.00          120fi.0l»  llo.l4        -  001 
ERROR                      113               1800.47              12. (ill 
TOTAL                     149  3243.70 

Dependent  Variable:  Retention  Test  (Regular  Enrolled  Students). 

Source  DF                   SS  MS  F  p 

TREAT  5  Ho.20  23.04  3.71  <01 

Pretest  1  1373.01  1373.01  221.10  <.001 

Posttest  1  841.42  841.42  135.50  <.001 

ERROR  142  881.80  6.21 

TOTAL  149  3211.44 

Question  #4 

The  following  measures  were  performed:  For  the  pretest  measure,  one  way  ANOVA; 
posttest,  one-way  ANCOVA  using  pretest  as  the  covariate;  retention  test,  one-way  ANCOVA 
using  pretest  and  posttest  as  covariates. 

For  Developmental  Studies  students,  no  treatment  differences  were  found  in  pretest 
and  posttest  scores.  There  was  a  significant  difference  in  retention  test  scores.  Tukey's 
Studentized  Range  Test  indicated  students  who  received  print  only  scored  better  than  those 
who  received  closed-captioned  video. 

For  Regular  Enrolled  students,  no  treatment  differences  were  found  in  pretest  and 
posttest  scores.  There  was  a  significant  difference  in  retention  scores.  Tukey's  Studentized 
Range  Test  indicated  students  who  received  closed-captioned  treatments  scored  significantly 
higher  than  those  students  who  received  print  only  treatment. 
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ANOVAs  &  ANCOVAs  for  Question  4 


Dependent  Variable:  Retention  Test 


(Developmental  Studies  Students). 


Source  DF 

TREAT  2 

Pretest  1 

Posttest  1 

ERROR  70 

TOTAL  74 


SS  MS  F 

68.59  34.29  4.94 

148.08  148.08  21,35 

313.20  313.20  4t.l6 

485.52  6.94 
1015.39 


<.01 

<.001 

<.001 


P 


Dependent  Variable:  Retention  Test 


(Regular  Enrolled  Students) 


Source  DF 

TREAT  2 

Pretest  1 

Posttest  1 

ERROR  70 

TOTAL  74 


SS  MS 

60.03  30.01 

597.81  597.81 

346,88  346.88 

404.27  5.78 
1408.99 


5.20 
103.51 
60.06 


<.01 

<.001 

<.001 


P 


Question  #5 

The  following  measures  were  performed:  For  the  pretest  measure,  one-way  ANOVA; 
posttest,  one-way  ANCOVA  using  pretest  as  the  covariate;  retention  test,  one-way  ANCOVA 
using  pretest  and  posttest  as  covariates.  No  treatment  differences  were  found  in  pretest, 
posttest,  and  retention  test  scores  for  both  Developmental  Studies  and  Regular  Enrolled 
students. 


Examination  of  data  from  baseline  measures  to  determine  differences  between 
Developmental  Studies  college  students  and  Regular  academic  core  students  revealed 
Developmental  Studies  students  knew  less  about  Civil  War  history  than  their  peers.  These 
students  would  appear  to  have  less  command  of  information  expected  of  students  who  enter 
college. 

This  study  suggests  that  using  different  instructional  media  may  not  be  effective  in 
bringing  Developmental  Studies  students  to  the  level  of  their  peers.  Even  though  gains  were 
made  in  posttest  and  retention  test  scores  in  these  students,  they  never  caught  up  in  level  of 
achievement  with  their  academic  core  peers.  None  of  the  six  uses  of  instructional  media  (i.e.. 
closed-eaptioned  video,  sound  video,  muted  closed-captioned  video,  audio  media,  audio  and 
printed  media,  and  printed  media)  was  successful  in  raising  posttest  and  retention  test 
scores  of  reading  deficient  students  equivalent  of  their  peers.  A  body  of  research  in  learning 
disabilities  suggest  these  children  never  catch  up  to  their  nondisabled  peers,  no  matter  what 
interventions  are  given.  This  research  with  students  having  reading  deficts  appears  *•> 
support  those  findings  with  learning  disabled  children.  However,  after  examining  uiifarences 
within  groups,  results  do  not  give  a  clear  picture  explaining  why  Developmental  Studies 
students  do  not  learn  or  retain  as  much  information  as  their  Regular  college  peers.  These 
results  might  suggest  students  who  have  reading  deficits  prefer  instructional  strategies  that 
require  one  sensory  mode  when  using  cognitive  skills  required  in  reading,  while  those  who  do 
not  have  deficits  can  more  easily  utilize  both  visual  and  auditory  senses  to  more  efficiently 
process  and  retain  information. 

Other  explanations  for  these  differences  can  be  derived  from  looking  at  the  data  for 
retaining  information  using  closed-captioned  video,  sound  video,  and  printed  media. 
Developmental  Studies  students  retained  more  information  when  given  printed  text.  They 
performed  worse  when  learning  through  closed-captioned  video.  However,  the  opposite 
happened  for  Regular  academic  core  college  students.  They  retained  more  learning  through 
closed-captioned  video  than  print  media. 
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These  results  suggest  more  is  involved  than  sensory  overload.  If  that  were  the  case, 
Developmental  Studies  students  would  not  be  significantly  different  from  their  peers  in  their 
posttest  and  retention  test  scores  for  those  treatments  involving  only  visual  or  auditory 
learning.  This  was  not  the  case.  Perhaps  Developmental  Studies  students  are  deficient  in 
metacognitive  skills  of  "knowing  how  to  learn."  Research  supports  that  children  with 
learning  disabilities  lack  metacognitve  skills  essential  in  reading;  yet,  the  majority  of  the 
Developmental  Studies  students  in  this  study  were  not  identified  as  having  a  learning 
disability.  These  findings  suggest  Developmental  Studies  students  may  share  some  of  the 
same  deficits  as  students  with  learning  disabilities  and  may  be  better  served  by  colleges 
and  universities  with  methods  and  materials  designed  for  students  with  learning 
disabilities. 

This  study  suggests  learning  through  multi-sensory  instructional  media,  such  as 
that  commonly  employed  in  academic  core  college  classrooms,  is  more  difficult  for  the 
Developmental  Studies  students.  However,  the  finding  that  Developmental  Studies 
students  who  received  muted  closed-captioned  treatment  retained  information  more  than 
their  same  peers  who  received  other  treatments  suggests  that  two  learning  modalities  (i.e., 
printed  word  and  visual)  are  effective  and  that  the  addition  of  the  third  modality  (i.e., 
sound)  may  interfere  with  learning  and  retention. 

More  research  should  be  conducted  to  determine  why  closed-captioned  video  appears 
to  be  the  most  effective  instructional  media  for  learning  retention  with  students  who  do  not 
have  reading  deficits.  The  results  of  this  study  seem  to  have  identified  another  factor  that 
can  make  closed-captioned  video  highly  successful  for  one  group  and  not  effective  for  another. 
Of  particular  importance  may  be  the  speed  of  the  prompt  rate.  These  prompt  rates  are 
designed  for  hearing  impaired  persons  who  are  considered  to  be  competent  readers. 
Therefore,  a  person  deficient  in  reading  skills  may  not  be  able  to  keep  up  with  a  fast  prompt 
rate,  could  become  easily  frustrated,  and,  therefore,  not  learn.  This  research  needs  to  he 
replicated  with  prompt  rates  modified  to  more  closely  match  reading  level  and 
comprehension  rate  assessed  in  students  with  deficient  reading  skills. 

The  use  of  technology  in  the  classroom  is  being  promoted  in  education  at  all  levels; 
however,  this  study  clearly  points  out  that  introduction  of  technology  can  have  detrimental 
effects  when  not  closely  examined.  Using  closed-captioned  video  is  supported  through  this 
research  as  being  an  effective  instructional  method  for  students  without  reading  deficits. 
With  more  research,  this  could  also  prove  true  for  those  who  struggle  to  learn  through 
reading,  such  as  students  in  Developmental  Studies,  in  special  education  programs,  and  in 
Chapter  I  reading  programs. 

At  this  time  we  do  not  know  why  all  students  did  not  respond  positively  to  closed- 
captioned  videotape.  Therefore,  the  results  of  this  study  should  be  interpreted  cautiously. 
More  research  is  needed  with  a  larger  number  of  students  with  reading  deficits. 
Additionally,  closed-captioned  video  technology  needs  to  be  responsive  to  allow  for  more 
manipulation  in  order  to  more  clearly  examine  its  power. 
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Gender 


Kuce 


Female 
Male 

Black 
White 
Other 


Mean 

Standard  Deviation 


Table  1:  Demographic  Data 

Developmental  Studies  Regular 

97  128 

63  22 


67  17 

81  130 

2  3 

21.73  24.59 

5.78  6.27 
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TABLE  2:  Group,  Treatment,  and  Group  by  Treatment  Means 


Pretest  Posttest  Retention  Test 

GROUP 

Dev.  Studies  11.22a  16.99  15.17 

3.22b  4.29  3.77 

Regular  15.73  22.96  20.52 

4.07  4.67  4.64 

TREATMENT 

CC  12  70  20.00  17.72 

4.40  5.67  5.70 

V  13.18  19.86  17.78 

3.95  4.63  4.69 
CM             13.24  20.00  18.22 

3.94  5.03  4.00 

A  12.94  19.04  17.14 

4.21  5.54  5.96 
AP             14.20  20.74  18.42 

4.53  6.43  5.49 

P  12.92  20.22  17.80 

3.96  4.92  3.92 
GROUP  by  TREATMENT  (Developmental  Studies) 

CC  10.28  16.24  14.12 

3.16  4.66  4.17 

V  11.08  16.92  15.00 

2.56  3.62  3.32 

CM  11.64  16.96  16.68 

2.69  3.49  2.98 

A  10.56  16.72  13.72 

3.58  3.84  3.79 

AP  11.84  16.40  15.08 

3.52  5.34  4.25 

P  11.92  18.72  16.44 

3.58  4.47  3.33 

GROUP  by  TREATMENT  (Regular) 

CC  15.12  23.76  21.32 

4.18  3.82  4.68 

V  15.28  22.80  20.56 

4.02  3.56  4.22 

CM  14.84  23.04  19.76 

4.37  4.49  4.33 

A  15.32  21.36  20.56 

3.39  6.06  5.82 

AP  16.56  25.08  21.76 

4.22  4.07  4.49 
P             13.92  21.72  19.16 

4.13  4.98  4.06 

a       Mean  Score  Standard  Deviation 

CC     Closed  Captioned      V      Video  Only  CM    Muted  Closed  Captioned 

A      Audio  AP    Audio  &  Printed  Text        P      Printed  Text 
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Figure  1:  Grouping  Chart 
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Figure  2:  Group  Comparison  of  Three  Measures 


Figure  3:  Treatment  Comparison  of  Developmental  Studies  Students'  Measures 
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Introduction 

Hypertext  is  a  way  of  organizing  information  using  nonlinear  text.  Hypertext 
provides  different  ways  from  traditional  text  to  access  and  manipulate  information.  The 
nonlinear  characteristics  of  hypertext  seem  to  change  the  ways  people  interact  with 
information  and  knowledge.  This  may  cause  learning  problems.  However,  little  empirical 
research  has  examined  the  results  of  hypertext  and  human  interaction. 

One  question  that  researchers  are  currently  exploring  is  "How  can  we  structure 
hypertext  for  maximum  learning?"  Based  on  theories  of  cognitive  style  and  research  in 
aptitude-treatment  interaction,  Witkin  et  al.  (1977)  suggested  that  educators  might  be  able 
to  adapt  instructional  treatment  to  the  learning  needs  of  field-dependent  and  field - 
independent  learners.  Therefore,  there  is  reason  to  examine  whether  different  linking 
structures  within  hypertext  environments  will  have  differential  effects  on  learners  with 
different  cognitive  styles.  Such  examination  may  indicate  that  it  may  be  necessary  to 
provide  access  constraints  to  serve  as  learning  guidance  in  order  to  make  hypertext  effective 
for  learners  with  different  cognitive  styles. 

The  purpose  of  this  study  was  to  determine  the  implications  of  content  organization 
and  cognitive  style  with  regard  to  the  design  of  hypertext.  Specifically,  the  study  examined 
the  effects  of  linking  structure  and  field-dependency  and  -independence  on  performance,  as 
well  as  attitudes. 

Linking  Structures  in  Hypertext  Environment 
The  five  linking  structures  used  in  this  study  are  based  on  Jonassen's  (1986)  three 
levels  of  hypertext  structure  (node-link  hypertext,  structured  hypertext.,  hierarchical 
hypertext)  and  include  linear  linking  structures,  hierarchical  linking  structures, 
hierarchical-associative  linking  structures,  associative  linking  structures,  and  random 
linking  structures.  Hierarchical  linking  and  associative  linking  are  most  frequently 
advocated  in  the  literature  (Gaines,  B.  R.  &  Vickers,  J.  N.,  1988;  Smith,  Weiss  &  Ferguson, 
1988). 

The  five  levels  of  hypertext  structure  used  in  this  study  will  be:  Linear  (in  which  the 
nodes  are  linked  linearly);  Hierarchical  (in  which  the  linking  emphasizes  the  relationship  of 
type  and  part  and  the  representation  of  hierarchies);  Hierarchical-Associative  (in  which  each 
node  set  is  structured  hierarchically  and  in  which  random  access  and  direct  access  to  any 
node  set  is  provided);  Associative  (in  which  the  linking  provides  random  access  and  direct 
access  to  any  node  in  the  hypertext);  and  Random  (in  which  any  node  can  be  linked  to  any 
other  node  directly).  Study  of  different  linking  structures  could  be  of  benefit  to  instructional 
designers  who  develop  materials  for  a  variety  of  learners  with  different  cognitive  styles,  e.g., 
field-dependence  and  field-independence. 

Individual  Differences  and  Field  Dependence  and  Independence 

According  to  Goodenough  (1976),  field-independent  (Fl)  individuals  adopt  an  active 
approach  toward  learning  while  field-dependent  (FD)  individuals  adopt  a  passive  approach 
toward  learning.  In  classroom  learning  situations,  FD  learners  appear  to  demonstrate 
poorer  structuring  abilities  than  FI  learners  (Goodenough,  1976).  The  FD  learners  display 
certain  distinctive  characteristics.  For  instance,  FD  students  tend  to  depend  on  and  be 
highly  influenced  by  "authority  figures"  and  are  less  able  than  FI  students  to  generate 
alternative  functions  for  elements  or  items  used  in  a  familiar  way  (Witkin  1977,  1979,  1981). 

In  this  study,  it  is  hypothesized  that  more  structured  hypertext  instruction  provides 
an  organizational  aid  to  learning,  especially  for  FD  learners.  In  contrast,  when  the  text  is 
less  structured*  organization  must  be  provided  by  each  learner,  and  would  therefore  be  loss 
facilitative  for  field-dependent  learners.  Furthermore,  because  FD  individuals  tend  not  to 
provide  organization  internally,  FD  subjects  should  have  particular  difficulty  with  less 


structured  sequences  that  contain,  for  example,  associative  and  random  hypertext  links. 
Thus,  it  is  expected  that  FD  learners  will  recall  fewer  concepts  than  FI  learners  in  learning 
from  a  hypertext-based  lesson  with  associative  and  random  links. 

The  Study 

The  purpose  of  this  study  was  to  examine  the  interactions  and  effects  of  hypertext 
linking  strategies  and  field-dependence  and  field-independence  (FDD  on  recall  performance 
and  attitudes  toward  learning  in  a  nonlinear  hypertextual  environment. 

Research  Questions 

The  research  questions  were  as  follows: 

1 .  Is  subjects1  performance/attitudes  significantly  predicted  by  linking  structure, 
FDI,  and  their  interaction? 

2.  Is  the  interaction  of  linking  structure  and  FDI  significant? 

3.  Do  the  five  linking  structures/attitudes  differ  in  subjects'  average  performance 
scores  with  FDI  held  constant? 

4.  Does  FDI  relate  to  subjects1  performancev  with  linking  structures  held 
constant? 

Independent  Variable 

The  two  independent  variables  in  this  study  were  linking  structure  type  and 
cognitive  style.  The  independent  variable  of  linking  structure  type  had  five  levels:  linear 
linking,  hierarchical  linking,  hierarchical-associative  linking,  associative  linking,  and 
random  linking.  The  second  independent  variable  was  field-dependence/independence,  as 
determined  by  results  of  the  Group  Embedded  Figures  Test  (GEFT)  (Witkin,  et  al.,  1971). 
Dependent  Variable 

The  primary  dependent  variables  included  in  this  study  were  the  learners1 
performance  and  attitudes.  Performance  was  measured  by  a  posttest  that  elicited  recall  of 
verbal  information.  Attitude  was  measured  by  an  attitude  survey.  Both  the  performance 
posttest  and  attitude  questionnaire  were  created  by  the  researcher  and  reviewed  by  a  content 
expert  and  two  instructional  design  specialists. 

Instructional  Materials 

Five  treatments  were  administered  to  learners  through  a  self-paced,  computer-based 
lesson  in  a  hypertext  environment.  The  contents  of  instructional  materials  for  the  five 
treatment  groups  were  based  upon  "Chinese  Politics"  which  was  developed  by  Kenneth  C. 
Wedding  (1991),  The  content  was  about  the  people,  organizations,  places  and  events  involved 
in  the  Tiananmen  Square  Event  of  June  4,  1989,  in  Beijing.  The  content  units  for  the  five 
treatments  were  identical,  but  each  employed  a  different  linking  strategy  using  linear 
linking  structures,  hierarchical  linking  structures,  hierarchical-associative  linking 
structures,  associative  linking  structures,  or  random  linking  structures. 

Hypertext  with  Linear  Linking  Structures 
In  linear  hypertext,  nodes  are  linked  linearly,  allowing  the  student  to  move  to  the 
next  or  to  the  previous  node  only  (see  Figure  1).  This  linear  structure  is  similar  to  most 
traditional  computer-assisted  instruction,  and  thus  serves  as  the  control  structure  for  this 
study.  An  example  would  be;  a  user  selects  one  "Event"  from  the  main  menu  and  reads  the 
information  under  the  category  of  "Events".  When  the  user  finishes  reading  the  "Event"  or 
decides  not  to  read  anymore  in  "Events,"  he  must  go  back  to  the  main  menu  and  select 
another  category. 
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Figure  1.  Hypertext  with  linear  linking 

Hypertext  with  Hierarchical  Linking  Structure 
In  hierarchical  hypertext,  a  node  at  one  level  can  only  access  nodes  directly  below  or 
above  it  (see  Figure  2).  An  example  would  be:  A  user  moves  along  the  hierarchical  structure 
and  reads  the  information  in  the  node  of  "Cultural  Revolution".  When  the  user  finishes 
reading  "Cultural  Revolution",  he  may  go  back  to  the  node  of  the  "Early  History"  which  is  the 
parent  node  of  the  "Cultural  Revolution".  Or,  the  user  may  go  to  "Great  Leap  Forward",  the 
other  child  node  of  "Early  History",  which  is  next  to  the  "Cultural  Revolution". 


Organizations 


Cultural  Great  Leap 


Revolut  ion  Forward 

Figure  2.  Hypertext  with  hierarchical  linking 

Hypertext  with  Hierarchical-Associative  Linking  Structures 
In  hierarchical-associative  hypertext,  nodes  are  basically  linked  in  a  hierarchical 
way.  The  associative  links  in  this  structure  are  provided  from  some  nodes  to  other  nodes 
that  are  referential  to  them.  That  is,  in  hierarchical -associative  hypertext,  a  node  at  one 
level  can  access  nodes  that  are  directly  below  or  above  it  or  a  node  associated  with  it  (see 
Figure  3).  This  type  of  linking  combines  associative  and  hierarchical  linking.  For  example,  a 
user  moves  along  the  hierarchical  structure  and  reads  the  information  in  the  node  "Cultural 
Revolution."  He  does  not  know  the  name  "Mao  Zedong"  which  appears  in  this  node.  He  may 
click  on  the  name  "Mao  Zedong"  to  go  to  the  node  "Mao  Zedong11  to  learn  about  him. 
However,  when  the  user  finishes  reading  "Mao  Zedong",  he  must  go  back  to  the  node 
"Cultural  Revolution." 
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Figure  3.  Hypertext  with  hierarchical-associative  linking 

Hypertext  with  Associative  Linking  Structures 
Similar  to  hierarchical-associative  linking,  associative  linking  is  based  upon  a  global 
hierarchical  structure.  Unlike  hierarchical-associative  linking,  which  requires  the  user  to 
return  to  the  original  hierarchical  node  from  the  referential  node,  associative  linking  allows 
the  user  to  link  to  the  referential  node  and  stay  in  the  node  set  to  which  the  referential  node 
belongs.  For  example,  a  user  moves  along  the  hierarchical  structure  and  reads  the 
information  in  the  "Cultural  Revolution"  node.  He  does  not  know  the  name  "Mao  Zedong" 
which  appears  in  this  node.  He  may  click  on  the  name  "Mao  Zedong"  to  go  to  the  node  "Mao 
Zedong"  to  learn  about  him.  While  reading  in  the  node  "Mao  Zedong,"  he  does  not 
understand  the  term  "Chinese  Communist  Party."  He  may  then  click  on  the  term  "Chinese 
Communist  Party"  to  go  to  the  node  containing  information  on  the  "Chinese  Communist 
Party"  to  learn  about  the  organization.  Alternately,  he  could  want  to  stay  in  the  node  set  of 
"Mao  Zedong"  (People)  to  learn  about  other  people  in  this  node  set.  (See  Figure  4.) 


Figure  4.  Hypertext  with  associative  linking 


Hypertext  with  Random  Linking  Structures 
Random  hypertext  allows  any  node  to  be  linked  to  any  other,  regardless  of  whether 
the  nodes  are  referential  to  each  other  or  not.  For  example,  a  user  can  jump  from  the  node 
about  Beijing  (Places)  to  the  node  about  the  Chinese  Communist  Party  (Organizations)  or 
jump  from  the  Cultural  Revolution  (Events)  node  to  the  node  about  Deng  Xiaoping  (People) 
directly,  (See  Figure  5.) 


Beijing 


Deng  Xiaoping 


Figure  5.  Hypertext  with  random  linking 

Subjects 

The  subjects  were  139  undergraduate  students  enrolled  in  the  College  of  Education 
at  The  University  of  Texas  at  Austin.   Participation  in  the  study  was  on  a  voluntary  basis. 
Subjects  were  randomly  assigned  to  one  of  the  five  condition  groups. 
Procedures 

A  brief  introduction  to  the  research  study  was  presented  to  students.  The  GEFT  was 
administered  by  the  researcher  during  the  class  immediately  after  the  students  signed  up  for 
the  experiment.  Field-dependence  scores  were  recorded.  Subjects  were  then  assigned 
randomly  to  the  computer  running  their  group  treatment  on  it.  The  researcher  introduced 
the  subject  of  the  lesson  and  the  actual  software.  Subjects  were  informed  of  the  grading 
policy  and  time  was  also  provided  for  any  questions  on  operating  the  program  that  the 
subjects  had. 

At  the  end  of  the  instructional  time,  the  performance  posttest  was  administered. 
Upon  completion  of  the  test,  an  attitude  questionnaire  was  distributed. 

Data  Analysis 

A  regression  analysis  test  was  chosen  to  detect  the  presence  of  any  aptitude- 
treatment  interaction.  According  to  Pedhazur  (1973),  the  advantage  of  using  regression 
analysis  is  that: 

MR,  Multiple  Regression,  is  applicable  to  designs  in  which  the  variables 
are  continuous,  categorical,  or  combinations  of  both,  thereby  eschewing 
the  inappropriate  or  undesirable  practice  of  categorizing  continuous 
variables  (such  as  designating  individuals  above  the  mean  as  high  and 
those  below  the  mean  as  low)  in  order  to  fit  them  in  what  is  considered, 
often  erroneously,  an  ANOVA  design  (p.7). 


4f)H 


In  this  regression  analysis,  a  continuous  value  of  cognitive  style  was  to  describe  as 
many  different  types  of  subjects  as  there  were  observed.  The  purpose  of  the  regression 
analysis  is  to  reveal  whether  or  not  the  group  regressions  (e.g.,  posttest  scores  on  GEFT 
scores)  are  homogeneous. 

Discussion  of  Results 
This  section  will  discuss  the  specific  results  of  this  study  as  they  related  to  the 
research  questions  and  hypotheses  posed  at  the  beginning  of  this  research  document.  Since 
there  were  some  significant  findings  as  a  result  of  data  analysis,  conclusions  based  upon  the 
results  will  be  suggested. 
Performance 

Research  Question  1:  Was  subjects'  performance  significantly  predicted  by  linking  structurr 
types,  FDI,  and  their  interaction? 

Discussion.  The  results  of  this  study  show  that  14.5%  of  the  variance  of  performance 
can  be  explained  by  linking  structure  type,  FDI,  and  their  interaction.  Since  research  on 
hypertext  is  rich  in  theory  and  poor  in  actual  study  results  (Jonassen,  1986),  some  scholars 
have  called  for  research  on  the  structure  of  hypertext,  proposing  different  types  of  linking 
structure  models.  Little  research  has  investigated  the  effects  of  linking  structure  type  and 
cognitive  style  on  performance  in  hypertext  environments.  The  findings  of  this  study  provide 
initial  evidence  on  this  issue.  Linking  structure  types  and  cognitive  style  and  their 
interaction  have  significant  effects  on  students'  performance. 

Research  Question  2:  Was  the  interaction  of  linking  structure  type  and  FDI  significant? 

Discussion.  Research  on  cognitive  style  suggests  that  consideration  of  style  may  be 
related  to  improved  performance  (Salmon,  1984;  Keller,  1983).  It  was  the  expectation  of  this 
study  that  consideration  of  these  two  variables  together  would  produce  significant  positive 
effects  on  performance. 

It  was  expected  that  subjects  with  less  field  dependency  would  perform  better  in 
more  structured  instructional  environments  whereas  subjects  who  were  more  field 
independent  would  perform  similarly,  regardless  of  the  linking  structures  employed.  The 
observations  do  not  support  these  expectations.  Rather,  findings  are  more  similar  to  those 
reported  by  Canino  &  Cicchelli  (1988)  and  MacNeil  (1980).  These  researchers  found  no 
interaction  of  treatments  and  cognitive  styles  upon  subjects'  performance. 
Research  Question  3:  Did  the  five  linking  structures  differ  in  average  subject  performance, 
with  FDI  held  constant? 

Discussion.  Research  has  shown  that  subjects  who  are  more  field-dependent  will 
perform  better  when  receiving  more  structured  instruction  (Mandler,  1967;  Meshorer,  1969; 
Clark,  1982,  1984,  1987).  The  results  of  this  study,  however,  do  not  support  these  findings. 
There  was  no  significant  difference  found  in  performance  between  subjects  working  in  a  more 
structured  environment  and  a  less  structured  environment. 

Research  Question  4:  Was  FDI  related  to  subjects1  performance  with  linking  structures  held 
constant? 

Discussion.  The  results  of  this  study  reveal  that  the  subjects  with  higher  scores  on 
the  GEFT  outperform  those  who  score  lower  on  the  GEFT  regardless  of  instructional 
treatment.  The  results  prove  consistent  with  the  studies  of  Carrier,  Davidson,  Higdon,  and 
Williams  (1984),  Carrier,  Joseph,  Krey,  and  LaCroix  (1933),  and  Frank  (1983).  These 
researchers  claimed  that  cognitive  style  influences  performance  in  general. 
Attitude 

Research  Question  1:  Was  subjects'  performance  significantly  predicted  by  linking  structure 
types,  FDI,  and  their  interaction? 


Discussion.  The  significant  results  of  the  full  model  regression  analysis  show  that 
subjects'  attitudes  could  be  predicted  by  linking  structure  type,  cognitive  style,  and  their 
interaction.  Some  researchers  have  called  for  research  on  the  structure  of  hypertext  and 
different  proposed  types  of  linking  structures  (Jonessen,  1986).  Little  research  has 
investigated  the  relationship  between  linking  structures  and  cognitive  styles  upon  attitude  in 
hypertext.  The  findings  of  this  study  provide  initial  evidence  on  this  issue  that  linking 
structure  types  and  cognitive  style  and  their  interaction  have  significant  effects  on  students' 
attitude  when  learning  in  a  hypertext  environment. 

Research  Question  2:  Was  the  interaction  of  linking  structure  and  FDI  significant? 

Discussion.  Research  on  cognitive  style  suggests  that  instruction  designed  with 
consideration  of  cognitive  style  may  be  related  to  improved  attitude  (Clark,  1982,  Salomon, 
1984,  Keller,  1983).  It  was  the  expectation  of  this  study  that  consideration  of  these  two 
variables  together  would  produce  significant  positive  results  upon  attitude. 

It  was  expected  that  subjects  with  lower  field  dependency  would  have  more  positive 
attitudes  when  using  less  structured  instruction,  while  subjects  who  are  more  field 
independent  would  prefer  more  structured  instruction.  The  results  of  this  study  support  the 
findings  from  Salomon  (1984),  Keller  (1983)  and  Clark  (1982).  Clark  (1982)  indicated  that 
field  independent  individuals  tend  to  be  more  self- motivated  and  have  greater  expectations  of 
achievement.  Although  they  may  learn  effectively  from  both  structured  and  unstructured 
methods,  they  may  believe  that  more  structured  methods  are  more  efficient  in  facilitating 
learning,  so  they  may  prefer  more  structured  instruction.  However,  field-dependent 
individuals'  motives  for  achievement  tend  to  be  low.  They  may  perceive  more  demanding 
methods  as  providing  more  freedom  for  students  to  control  learning  time  and  effort  level. 
With  structured  methods  they  can  invest  less  effort,  and  their  failures  are  less  visible. 
Therefore,  they  like  less  structured  methods,  which  seem  to  make  their  lives  easier.  The 
results  of  the  calculations  support  the  expectation  that  field- independent  individuals  prefer 
structured  instruction. 

Research  Question  3:  Did  the  five  linking  structures  differ  in  average  attitude  score  with 
FDI  held  constant? 

Discussion,  The  findings  for  this  question  indicated  that  subjects  tended  to  prefer 
the  hierarchical  structures  and  the  hierarchical-associative  structures  more  than  linear 
structures. 

Research  Question  4:  Was  FDI  related  to  subjects'  attitudes  with  linking  structures  held 
constant? 

Discussion.  The  results  indicate  that  students  who  tend  toward  field-independence 
have  more  positive  attitudes  about  learning  about  "Chinese  Politics"  from  the  hypertext 
system  used  for  this  study  than  those  who  tend  toward  field-dependence. 
Support  Data 

Although  no  formal  hypotheses  were  formulated  for  the  following  analyses,  there 
were  areas  related  to  the  previously  stated  hypotheses  that  served  to  support  or  elaborate 
upon  the  findings  associated  with  them. 
Prior  Knowledge  of  Hypertext  Systems 

When  the  effect  of  prior  knowledge  of  hypertext  systems  was  taken  into  consideration 
in  reviewing  subjects'  performance  and  attitudes,  results  showed  that  prior  knowledge  had 
no  main  effect  on  either  performance  or  attitudes.  Regression  analysis  also  showed  that 
prior  knowledge  c**~ot  significantly  predict  subjects*  performance  or  attitudes. 
Prior  Knowledge  l  Chinese  Politics 

When  the  effect  of  prior  knowledge  of  subject  content  v/as  taken  into  consideration  in 
reviewing  subjects  performance  and  attitude,  the  results  showed  that  prior  knowledge  had  a 
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main  effect  on  both  performance  and  attitude.  The  more  the  students  knew  about  the 
subject,  the  better  they  performed  on  recall  of  the  content  (p  <.01).  The  more  the  students 
knew  about  the  subject,  the  better  attitudes  they  had  toward  the  instructional  materials  (p 
<05). 

Conclusion 

To  summarize,  the  results  of  this  study  indicate: 

1.  The  performance  of  these  subjects  can  be  predicted  by  linking  structures,  cognitive 
style,  and  their  interaction. 

2.  The  performance  of  these  subjects  cannot  be  predicted  by  the  interaction  of  linking 
structure  types  and  cognitive  style. 

3.  There  is  no  difference  in  subjects'  performance  when  learning  from  hypertext  systems 
with  different  linking  structures. 

4.  Field-independent  subjects  outperform  field-dependent  subjects  overall, 

5.  The  attitudes  of  the  subjects  can  be  predicted  by  linking  structure  types,  cognitive 
styles,  and  their  interaction. 

6.  The  attitudes  of  the  subjects  can  be  predicted  by  interaction  of  the  linking  structures 
and  cognitive  styles. 

7.  Students  like  hierarchical  structures  and  hierarchical-associative  structures  more  than 
linear  structures  in  hypertext  environments. 

8.  Field-independent  subjects  tend  to  have  better  attitudes  than  field-dependent  subjects. 

To  summarize,  the  results  of  this  study  failed  to  support  the  hypotheses  for 
interaction  between  linking  structure  types  and  cognitive  styles  for  both  performance  and 
attitude,  including  a  main  effect  of  linking  structure  upon  performance.  The  lack  of  support 
for  these  hypotheses  may  have  been  due  to  methodological  rather  than  conceptual  causes.  It 
is  possible  that  specific  details  of  this  study  may  have  contributed  to  the  significant  findings 
in  this  research. 

Recommendations  for  Future  Research  on  Hypertext 

1.  Information  seeking  strategies  in  hypertext  should  be  investigated.  In  this  study, 
five  linking  structures  were  investigated.  All  the  subjects  were  taught  by  the  researcher 
about  how  to  navigate  through  the  program.  However,  important  questions  remain.  How  do 
users  learn  information  navigating  strategies,  and  how  do  they  apply  them  in  a  non- 
instructional  situation?  What  can  search  systems  do  to  assist  in  the  selection  find 
application  of  strategies?  The  use  of  index  menus  in  this  study  is  only  one  representation  of 
browse  strategies.  Other  manifestations  of  these  strategies  could  he  examined,  such  as 
embedded  menus  or  navigation  maps. 

2.  Qualitative  methods  can  be  adopted  by  hypertext  researchers.  In  situations  such 
as  these  where  complex  interactions  are  likely  to  occur,  a  qualitative  approach  with  which 
researchers  can  look  for  patterns  may  be  useful  in  building  hypotheses  for  qualitative 
studies.  One  approach  to  such  a  study  is  to  examine  individual  pathways  in  a  hypertext 
lesson.  By  focusing  on  paths  that  are  taken,  a  hypertext  instructional  system  might  be 
improved. 
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ABSTRACT 


A  pretestposttest  control  group  design  with  random  assignment,  together  with 
qualitative  data  collection  and  analysis,  was  used  to  investigate  whether  metacognitive, 
cognitive  and  affective-awareness  cues  embedded  in  a  hypermedia  program  could  facilitate 
college  students1  near  and  far  transfer  of  problem  solving  in  biology  learning.  It  was 
assumed  that  when  subjects  are  asked  to  explain  reasons  for  their  own  actions  during  the 
problem  solving,  they  must  engage  in  self-reflective  intermediate  processes  comparable  to 
metacognitive  processes  of  monitoring,  evaluating,  and  regulating  ongoing  problem  solving. 
Pour  treatment  groups  were  used:  one  received  metacognitive  cues;  one  received  cognitive 
cues,  one  received  affective-awareness  cues,  and  the  control  group  received  no  cues.  Results 
showed  that  subjects  in  the  metacognitive  group  performed  better  on  far  transfer  tasks 
than  all  other  groups.  The  qualitative  analysis  indicated  that  metacognitive-oriented 
questions  encouraged  students  to  "stop,  think  and  reflect"  on  their  problem  solving  processes 
which  in  turn  helped  students  understand  the  process  of  how  the  problems  were  solved. 
The  understanding  of  processes  students  went  through  to  solve  problems  helped  them 
transfer  what  they  learned  to  novel  situations. 
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Introduction 

One  of  the  primary  goals  of  education  is  to  enable  individuals  to  develop  skills  which 
allow  them  to  continue  self-education  throughout  their  lives.  In  the  past  15  years,  helping 
students  develop  self-regulated  learning  skills  has  been  commonly  known  as  helping 
students  develop  skills  of  learning  how  to  learn.  Self -regulated  learning  is  defined  as  the 
extent  to  which  the  students  are  metacognitively,  motivationally,  and  behaviorally  active 
participants  in  their  own  learning  processes  (Zimmerman,  1986). 

The  heart  of  self- regulation  is  developing  the  awareness  and  capacity  for  effective 
planning,  monitoring,  reflecting,  and  modifying  one's  own  learning  processes  (Corno  & 
Mandinach,  1983;  Brown,  Bransford,  Ferrara,  &  Campione,  1983).  Corno  and  Mandinach 
(1 983)  further  suggest  that  long-term  exercising  of  self- regulated  learning  activities  will  help 
students  at  all  levels  learn  how  to  learn  and  develop  school  aptitude  and  motivation.  It  is 
important  to  note  that  the  definition  of  self-regulated  learning  encompasses  metacognitive, 
cognitive,  and  affective  processes.  Flavell  (1976)  defines  metacognition  as  the  awareness 
and  active  monitoring  of  one's  cognitive  processes.  In  problem  solving,  the  metacognitive 
processes  monitor  the  selection  and  application  of  solution  processes  as  well  as  ongoing 
feelings  about  self  and  task.  A  metacognitively  aware  learner  is  able  to  "apply,  adapt 
and/or  modify  what  they  have  learned  to  new  tasks  and  across  different  situations*'  (Rowe, 
1988,  p.  228).  It  is  the  understanding  and  regulating  of  their  own  cognitive  processes  that 
enable  students  to  tackle  new  tasks,  solve  new  problems,  and  transfer  learning  across 
domains  (Pintrich,  Cross,  Kozma,  &  McKeachie,  1986;  Glaser,  1984;  Rowe,  1988;  Berardi- 
Colette,  1990;  Dominowski,  1990). 

Transfer  is  defined  as  the  process  in  which  students  use  previously  learned  knowledge 
and  skills  to  successfully  solve  problems  in  new  situations  (Gagne  &  Berliner,  1988).  It  can 
be  divided  into  two  levels:  near  and  far  transfer.  Near  transfer  refers  to  situations  in  which 
training  situations  and  problem  tasks  are  almost  identical  (Butterfield  &  Nelson,  1991; 
Salomon  &  Perkins,  1989).  Far  transfer  occurs  when  knowledge  and  skills  are  successfully 
applied  to  a  highly  novel  problem,  and  it  requires  creative  use  of  knowledge  and  strategies 
learned  (Clark,  1992). 

A  growing  body  of  literature  supports  the  notion  that  optimal  academic  performance  is 
strongly  tied  to  the  degree  of  self- regulation  the  learner  is  capable  of  exercising  (Zimmerman, 
1990;  Horkowski,  et.  al.,  1990;  Pintrich  &  DeGroot,  1990;  Corno,  1986).  As  a  result, 
considerable  attention  has  been  focused  on  the  phenomenon  of  self-regulated  learning. 
There  has  hern  much  progress  made  in  (1)  identifying  knowledge  structures  and  processes 
that  underlie  self-regulated  learning  and  (2)  recognizing  the  importance  of  a  motivational 
component  in  addition  to  metacognitive  and  cognitive  components  (McCombs  &  Marzano, 
1  W)U).  The  integrative  nature  of  the  concept  may  lead  to  increased  theoretical  complexity  of 
the  construct  and  a  greater  understanding  of  learning  processes.  However,  to  use  the 
concept  of  self- regulation  as  a  realistic  guide  for  everyday  classroom  situations,  there  is  a 
continued  need  for  theoretical  and  empirical  research  which  shows  how  each  component  of 
self  regulation  impart  learning  and  problem  solving.  Further  research  is  also  needed  to 
learn  more  about  the  role  affective-awareness  plays  in  contributing  a  better  problem  solving 
performance.  These  needs  provided  the  theoretical  basis  for  this  study. 

Computer  technology  is  thought  to  hold  great  potential  for  more  effective  teaching  of  self- 
regulated  learning  skills  (Henderson,  1986).  It  has  been  suggested  that  technology-based 
instruction  is  likely  to  result  in  citizens  who  know  how  to  learn  and  will  continue  to  learn  in 
the  future  if  the  program  used  emphasizes  self-discovery  of  new  ideas,  helps  learners  gain 
feedback  about  their  ideas,  and  encourages  them  to  assess  their  own  ideas  (Linn,  1983). 
Hypermedia  systems,  a  fairly  recent  advance  in  technology,  provide  more  potential  and 
advantages  to  learning  over  conventionally  programmed  CA1  due  to  their  greater  degrees  of 
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flexibility  and  capacity  for  individualized  instruction  (Bourne,  1990).  The  term  "hypermedia" 
is  defined  as  an  advanced  technology  that  combines  film,  video,  computer  graphics,  sound, 
music,  and  text  in  a  unified  information -delivery  system  centered  upon  the  personal 
computer  (Paske,  1990).  It  is  a  mixture  of  technologies  controlled  by  hypertext.  Hypertext  is 
the  term  that  is  used  to  describe  non-sequential  writing,  text  that  branches  and  allows 
choices  to  be  made  by  the  learners  (Paske,  1990).  In  a  hypermedia  system,  the  information 
can  be  organized  in  a  non-linear  manner  and  a  topic  can  be  explored  in  multiple  ways  (Spiro 
&  Jehng,  1990). 

In  addition  to  the  theoretical  problem  posed  earlier,  two  practical  problems  were  also 
addressed  in  this  study  that  involve  the  role  of  hypermedia  in  students'  learning.  They 
were:  (a)  how  each  component  of  self-regulated  learning  processes  (metacognitive,  cognitive 
and  affective  processes )  influenced  positive  near  and  far  transfer  of  problem  solving  skills; 
and  (b)  how  hypermedia  should  be  used  to  promote  self-regulated  learning  which  might  lead 
to  positive  transfer. 

Although  previous  research  has  shown  that  positive  problem  transfer  is  difficult  to 
obtain  (Thomas,  1974;  Gick  &  Holyoak,  1980),  studies  requiring  subjects  to  explain  reasons 
for  actions  while  solving  problems  have  consistently  shown  positive  transfer  effects  (Gagne 
and  Smith,  1962;  Berry  &  Broadbent,  1987;  Berardi-Coletta,  1990;  Stinessen,  1985).  These 
studies  typically  required  subjects  to  "state  reasons"  for  their  actions  during  the  problem 
solving.  Most  of  these  studies  dealt  with  college  students  in  solving  the  types  of  problems 
involving  puzzles.  They  all  found  successful  near  and  far  transfer  effects.  Most  of  their 
explanations  for  the  positive  effect  on  solution  transfer  have  been  limited  to  speculations 
that  verbalization  induced  more  planning  and  forced  subjects  to  think  (Gagne  and  Smith, 
1962),  and  verbalization  provided  the  metacognitive  experience  needed  for  successful 
transfer.  However,  it  was  not  clear  what  was  responsible  for  the  positive  transfer, 
verbalization  or  metacognitive  processes. 

Berardi-Coletta's  (1990)  study  answered  these  questions.  She  concluded  that  it  was  not 
verbalization  which  produced  the  positive  transfer  effect.  Rather,  metacognitive  processes 
enabled  subjects  to  understand  the  process  of  how  the  problem  was  solved,  which  led  to 
positive  transfer  (Berardi-Coletta,  1990).  Giving  reasons  for  one's  own  actions  involved 
learners  in  metacognitive  process  (Berardi-Coletta,  1990).  She  also  found  in  her  study  that 
negative  seif-evaluation  distracted  learners  from  focusing  on  the  task,  and  thus  produced 
negative  effects  on  transfer  performance.  Hov/ever,  she  did  not  explain  how  affective- 
awareness  influences  near  and  far  transfer  of  problem  solving.  Further,  it  was  not 
demonstrated  in  these  studies  if  the  use  of  self-explanation  cues  to  induce  metacognitive 
processes  which  were  effective  for  puzzle  problem  transfer  remain  effective  for  content-rich 
problem  transfer. 

It  is  also  important  to  investigate  whether  embedding  question  cues  in  hypermedia 
system  to  encourage  learners  to  self-explain  their  own  problem-solving  processes  is  effective 
for  content-rich  problem  transfer  because  technology  is  likely  to  become  an  increasingly 
common  component  of  learning  in  both  education  in  schools  and  business  training.  With  the 
advent  of  technology,  especially  hypertext  and  hypermedia  which  provide  high  levels  of 
flexibility  and  autonomy  for  learners  to  explore  and  construct  their  own  learning,  it  should  be 
possible  to  assist  learners  to  acquire  self-regulatory  processes  that  are  essential  for  effective 
transfer  of  learning, 

In  summary.  the  theoretical  needs  \o  explore  how  each  individual  component  of  self- 
regulated  learning  theory  impacts  learning  and  problem  solving,  together  with  the  practical 
nerds  to  investigate  how  hypermedia  systems  should  be  used  to  promote  self- regulated 
learning  and  positive  transfer  of  problem  solving,  provide  the  rationale  for  this  study. 

The  purpose  of  this  study  was  to  investigate  1)  whether  metacognitive,  cognitive,  and 
ufl'ertive  awareness  processes  provoked  by  self-explanation  cues  embedded  in  a  hypermedia 
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simulation  system  improves  immediate  near  and  far  transfer  problem  solving  in  biology 
learning;  2)  how  subjects  respond  to  metacognitive,  cognitive,  and  affective  cues;  and  3)  how 
metacognitive,  cognitive  and  affective  cues  influence  students'  task  design  in  the  hypermedia 
system  during  the  training. 

It  was  hypothesized  that  there  would  be  no  differences  among  groups  in  near  transfer. 
However,  subjects  who  received  metacognitive  cues  were  expected  to  perform  better  than 
subjects  who  received  cognitive  or  affective  cues  and  students  who  received  no  cues  in 
immediate  far  transfer  problem  solving,  Subjects  who  received  cognitive  and  affective  cues 
were  expected  to  perform  equally  well  on  immediate  far  transfer  of  biology  problem  solving. 
However,  subjects  in  cognitive  and  affective  groups  were  expected  to  perform  better  than 
subjects  in  control  groups  (no  cues). 


Methodology 
Experimental  Design 

This  study  involved  pretest-posttest  control  group  design  with  random  assignment, 
together  with  qualitative  data  collection  and  analysis.  There  were  three  treatment  groups 
and  one  control  group.  The  three  treatment  groups  were:  (1)  a  metacognitive  group  which 
responded  to  question  cues  from  the  computer  program  which  asked  subjects  to  plan  ahead 
of  time,  monitor  the  process,  evaluate  the  results  and  modify  task  design;  (2)  a  cognitive 
group  which  responded  to  question  cues  that  asked  subjects  to  explain  specific  rules,  goals, 
strategies  which  were  related  to  the  content  of  specific  problems);  and  (3)  an  affective- 
awareness  group  which  responded  to  question  cues  that  asked  subjects  to  explain  their 
concurrent  feelings,  or  to  compare  feelings  at  the  beginning  to  those  at  the  end.  Subjects  in 
the  control  group  did  not  receive  any  cues.  All  of  the  groups  received  the  same  level  of 
training,  which  differed  only  in  the  types  of  cues  on  which  training  focused.  The  locations  of 
the  cues  were  the  same  for  all  of  the  three  treatment  groups.  The  program  was  identical  for 
all  treatment  groups  except  for  the  specific  cues.  Appendix  1  lists  the  content  and  locations 
of  the  cues  for  all  three  groups. 

The  independent  variable  was  the  type  of  cues;  metacognitive  cues,  cognitive  cues, 
•  fectivc-awaroness  cues  and  no  cues.  The  dependent  variables  were  near  and  far  transfer 
of  problem  solving. 


Subjects 

Kighty  eight  pre-service  elementary  education  majors  enrolled  in  an  introductory  course 
in  biology  at  Purdue  University  volunteered  to  participate  in  this  st»*dy.  The  tolal  enrollment 
of  the  class  was  205,  with  195  females  and  10  males.  The  distribution  of  the  cIhhh  was 
approximately  one-third  juniors,  one-third  sophomores,  with  the  rest  a  mixture  of  freshman 
and  seniors.  The  study  was  an  extra  credit  project  option  provided  by  the  instructor. 
Originally  one  hundred  forty  students  signed  up  for  the  project.  But  only  eighty-eight  were 
included  in  the  data  analysis  due  to  several  technical  difficulties,  which  included  program 
hugs  and  system  compatibility  problems.  Among  eighty -eight  subjects,  eighty-four  of  I  hem 
were  females  and  four  of  them  were  males.  All  of  them  passed  content  tests  so  that  content 
knowledge  would  not  be  a  factor  that  influenced  the  performance.  Only  5%  of  the  subjects 
were  familiar  with  Macintosh  computers.  However,  experience  with  computers  was  not 
considered  a  concern  to  the  project, 
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Materials 

This  research  project  was  part  of  a  laboratory  experiment  which  had  been  found  to  be 
problematic  for  students  enrolled  in  the  class  in  the  past.  The  content  selected  for  the 
hypermedia-based  training  task  was  a  series  of  experiments  designed  by  the  students  to 
test  the  effects  of  light  and  moisture  on  isopod  behavior.  We  will  refer  to  these  as  the 
"student  tasks",  to  avoid  confusion  between  this  experiment  and  the  experiments  which  the 
students  were  designing.  To  solve  these  problems,  subjects  had  to  recognize  the  need  to 
control  variables  and  understand  random  behavior  to  verify  their  derived  solutions,  identify 
experiments  which  did  not  produce  useful  data,  and  repair  them.  It  has  been  suggested  by 
several  researchers  that  the  lack  of  metacognitive  reasoning  abilities  might  be  the  major 
factor  inhibiting  subjects  from  solving  this  type  of  problem  successfully  (Lawson,  Blake,  & 
Nordland,  1975;  Lawson,  1979;  Piaget,  1962;  Lawson  &  Nordland,  1976). 

Training  tasks  were  built  into  a  hypermedia  simulation  program  running  on  Apple 
Macintosh™  personal  computers  that  linked  Authorware™  (Macromedia  Inc)  and 
HyperCard™( Claris,  Inc.).  There  were  six  options  in  the  program  menu,  and  subjects  were 
given  the  freedom  to  select  their  own  content,  pace  and  paths.  The  program  was  developed 
by  the  authors  of  this  paper. 

Another  commercial  program,  Screen  Recorder™(Farallon,  Inc.),  was  also  running  at  the 
same  time  when  a  randomly  selected  sample  of  subjects  were  using  the  simulation  program. 
This  program  created  a  "record'*  of  everything  that  happened  on  the  screen,  including  the 
selected  subjects'  time  on  task,  sequence  of  their  selections,  design  of  the  tasks,  learning 
paths,  and  frequency  of  selection  of  each  option.  The  purpose  of  this  program  was  to  collect 
qualitative  data  on  how  the  different  cues  influenced  subjects'  responses  and  design  of  the 
experiments. 

The  test  of  near  transfer  measures  required  subjects  to  work  with  a  preset  experimental 
design  to  test  the  effect  of  moisture  on  isopod  behavior.  The  far  transfer,  however,  asked 
subjects  to  design  an  experiment  to  determine  which  of  three  possible  causes  (peer  influence, 
diet,  or  family  environment)  were  responsible  for  an  increase  in  hyperactivity  in  a 
hypothetical  elementary  school  class.  Both  near  and  far  transfer  tests  were  developed  by 
the  instructor  of  the  course.  Refer  to  Appendix  2  for  the  copy  of  the  test. 


All  of  the  students  met  for  the  lecture  once  per  week  for  50  minutes.  There  were  also 
nine  hands-on  laboratory  classes  with  about  25  students  per  classes.  When  registering  for 
the  lecture,  subjects  signed  up  for  the  lab  classes  based  on  their  schedules  for  the  semester. 
There  was  one  instructor  for  the  lecture  and  nine  teaching  assistants  (TAs)  for  the  lab 
divisions.  Each  hands-on  lab  division  met  for  two  90  minutes  sessions/week,  and  there 
were  always  two  teaching  assistants  present. 

The  project  was  divided  into  three  phases,  pre-treatment  (four  weeks),  treatment  ( two 
weeks)  and  post- treatment  (one  week).  The  pre-training  v/as  conducted  because  it  has  been 
suggested  by  Berry  and  Broadbent  (1984)  that  concurrent  verbalization  or  self-explanation 
may  only  have  a  significant  effect  when  accompanied  by  pre-task  training  on  how  to  respond 
to  the  questions. 

During  the  first  week  of  the  pre-tr  ^atment  phase,  TAs  were  introduced  to  the  project  by 
the  researchers.  The  purposes  of  the  project,  time  required  of  TAs  for  help,  and  overall  plan 
and  r-cope  of  the  project  were  all  explained.  They  were  also  asked  to  go  through  the 
computer  program  as  well.  The  researches  went  to  the  lecture  to  explain  the  project  to  all 
of  the  students.  The  policy  for  extra  credit  was  explained  as  well  as  the  learning  benefits 
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students  might  obtain  by  participating  in  the  project.  They  were  told  that  this  experience 
could  help  them  to  learn  to  solve  problems  better,  which  could  be  applied  to  other  course 
projects  they  would  be  assigned.  They  were  also  told  that  training  would  be  provided  by  the 
researchers  and  TAs  so  that  they  would  know  how  to  use  the  program  when  they  came  to 
the  computer  lab.  This  was  done  to  alleviate  possibilities  of  student  unfamiliarity  and/or 
anxiety  about  the  use  of  computers. 

During  the  second  and  third  week  of  the  pre-training  phase,  subjects  completed  a  hands- 
on  isopod  lab  experiment  during  their  regularly  scheduled  lab  divisions.  At  the  same  time, 
the  sign- up  sheets  for  the  computer  lab  schedules  were  posted  so  that  subjects  would  have  3 
weeks  to  sign  up  for  the  computer  labs.  Each  subject  was  required  to  sign  up  for  two 
computer  sessions  with  2  hours  each  session.  TAs  were  instructed  by  the  researchers  on 
how  to  train  subjects  from  different  treatment  groups  to  respond  to  different  cues.  The 
instructions  for  the  training  were  given  to  all  of  the  TAs,  Students  were  randomly  assigned 
within  divisions  to  each  treatment  group  by  the  researchers,  and  the  TAs  were  also  informed 
of  the  names  within  each  treatment  group. 

During  the  fourth  week  of  the  pre-training  phase,  student  training  on  how  to  work  with 
the  programs  and  how  to  answer  cue  questions  were  conducted  by  the  researchers  and  TAs 
in  their  regular  lab  sessions.  The  random  assignment  within  divisions  gave  approximately 
7  subjects  per  treatment  group  within  each  division. 

During  the  treatment  phase,  eighteen  Mac  Ilex  computers  in  a  computer  lab  were  used. 
Each  computer  was  equipped  with  the  four  versions  of  the  program  corresponding  to  the 
three  treatment  groups  and  one  control  group.  Each  version  required  subjects  to  enter  their 
own  passwords  to  access  a  specific  treatment.  The  password  was  given  to  the  subjects 
during  the  pre-training  session.  By  assigning  the  passwords  to  each  student,  the 
treatments  were  effectively  isolated,  preventing  any  opportunities  for  mixing  the  treatments. 
The  researchers  supervised  the  lab  across  all  sessions  to  make  sure  that  everything  ran 
smoothly. 

During  the  post-treatment  phase,  subjects  were  informed  about  the  real  nature  of  the 
project  and  were  told  that  treatment  was  manipulated.  They  were  encouraged  to  practice 
with  different  versions  of  the  program  so  that  everyone  was  exposed  to  the  same  versions  of 
the  program  to  reduce  the  possible  effects  caused  by  the  manipulation  in  their  performance 
on  the  course  final  exams. 


Data  Collections  and  Analysis 
The  data  collection  was  divided  into  three  stages,  pretest  data  collection,  on  line  data 
collection  and  posttest  data  collection.  Three  weeks  before  the  treatment,  all  of  the  subjects 
who  signed  up  for  the  project  were  given  pretests  on  near  and  far  transfer  problem  solving. 
The  on-line  data  was  collected  by  Screen  Recorder™(Farallon,  Inc.)  running  at  the  same  time 
when  selected  subjects  were  working  in  the  program,  as  well  as  information  which  the 
program  itself  recorded  to  disk 

(student  responses  to  the  cues  and  time  on  task).  Three  subjects  from  each  treatment  group 
and  the  control  group  were  selected  randomly.  No  subjects  were  interviewed  during  the 
treatment  time. 

As  soon  as  the  subjects  finished  two  computer  lab  sessions,  they  were  Riven  the 
posttests  on  near  and  far  transfer.  The  content  was  exactly  the  same  for  pre  and  posttests 
except  that  the  orders  of  the  questions  had  been  changed.  Eight  subjects  from  each  of  the 
treatment  and  control  groups  for  a  total  of  16  were  selected  for  interviews  ri^ht  after  they 
finished  the  near  and  far  posttests.  Five  questions  were  used  to  direct  the  interview,  and 
the  same  questions  were  asked  across  groups.  They  were  1)  how  subjects  responded  to  the 
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different  cues;  2)  how  the  cues  affected  the  process  of  on-line  task  problem  solving,  3)  why 
some  subjects  did  not  respond  to  the  cues;  4)  whether  the  subjects  asked  themselves  the 
same  questions  during  the  transfer  problem  solving;  and  5)  how  the  experience  affected 
subjects'  confidence  and  task  interests.  Refer  to  Appendix  4  for  the  sample  protocols  from 
each  group. 

The  data  was  analyzed  both  quantitatively  and  qualitatively.  One-way  ANOVA  and 
post-hoc  Newman- Keuls  were  used  to  measure  posttest  differences  among  groups.  Paired 
sample  t-tests  were  used  to  measure  the  pre  and  posttests  differences  within  groups.  A 
random  sample  of  6  protocols  from  each  group  were  coded  for  the  cue  response  content  by 
two  independent  raters.  Interrater  reliability  was  checked  to  be  sure  that  subjects  in  the 
different  treatment  groups  were  in  fact  responding  to  the  type  of  information  that  was 
requested  in  the  question  cues.  The  qualitative  data  were  analyzed  within  each  group  to 
find  the  patterns  of  subjects'  responses  and  learning  processes  based  on  their  responses  to 
tl  e  interviewer,  and  the  student  task  designs  were  analyzed  through  a  constant  comparison 
process  for  any  subgroupings. 


Results 

Quantitative  Findings 

To  test  the  main  hypothesis  that  all  of  the  groups  would  perform  equally  well  on 
immediate  near  transfer  test,  and  that  the  metacognitive  group  would  outperform  all  of  the 
other  groups  in  immediate  far  transfer  test,  analysis  of  variance  (ANOVA)  was  used  to 
compare  performance  across  the  groups  on  near  and  far  transfer  measures.  Refer  to  Table  1 
for  the  means  and  standard  deviations  for  the  pre  and  post  measures  on  dependent 
variables  among  groups.  There  was  no  significant  mean  score  differences  among  groups  on 
the  pretest  of  near  or  far  transfer.  Neither  were  significant  mean  score  differences  found 
among  groups  on  posttest  near  transfer  and  the  average  time  spent  on  the  training  task. 
However,  the  ANOVA  for  post  far  transfer  tests  indicated  there  was  a  statistically 
significant  main  effect  among  groups:  F  (3,  84)  =  5,95,  p  <.001.  Student-Newman-Keuls 
(SNK)  was  chosen  for  post  hoc  paired  comparisons  contrasting  group  means.  Results  of  a 
post  hoc  SNK  showed  the  order  of  the  group  means  (in  descending  magnitude)  in  post  far 
transfer  (Refer  to  Table  2).  Means  which  were  not  underlined  by  the  same  line  were 
significantly  different  from  each  other.  The  finding  suggests  that  the  metacognitive  group 
outperformed  all  of  the  other  groups  in  posttest  far  transfer.  The  cognitive,  affective- 
awareness,  and  control  groups  were  not  significantly  different  from  each  other.  The  main 
hypotheses  were  supported. 


470 

<1 6  9 


Table  1:  Individual  group  means  and  standard  deviations  for  dependent 
variables  of  pre  and  posttests  on  near  and  far  transfer: 


Variable  . 

Group 

N 

Mean 

Std  Dev 

Pr<  *  _  M  o  o  *• 

/  V11CX.L1  VC 

9 1 

1  5  fi1  Q 

9  70fi 
j£ .  /  un 

90 

1  O .  OuU 

1  ftl 9Q ' 

1  LI  HI 

23 
z>  o 

1  R  9^Q 

Metacog. 

20 

15.925 

2.7012 

Prf*-F«r 

1   I  "    A  (XI 

Cncnif'ivp 

VU|^  1 1 1  l»l  V  c 

20 

1.950 

\_vUl  1  U  Ul 

1  .DO  1 

Metacog. 

20 

2.150 

1.785 

rUsL'liCal 

9  2 

9 

Coerntivp 

2 1 

16.69 

2.337 

Pnntrr.l 

94 

IUiUu 

21 

18.09 

1 .505 

Post-Far 

Affective 

22 

2.318 

1.728 

Cognitive 

21 

2.523 

1.364 

Control 

24 

1.708 

1.366 

Metacog. 

21 

3.476 

1.123 

Time 

Affective 

15 

77.46 

24.12 

Cognitive 

15 

70.00 

18.40 

Control 

17 

80.29 

33.79 

Metacog. 

17 

67.58 

18.12 

Table  2:  Results  of  the  Post  Hoc  SNK  contrasting  group  means  on  post  far 
transfer  test: 


Meta  Cognitive  Affective  Control 


M       3,476  2J24  LMS  UM 

The  differences  between  pre-post  mean  scores,  standard  deviations,  T  scores,  and  p 
values  on  near  and  far  transfer  within  groups  are  listed  in  Table  3.  The  results  indicate 
that  metacognitive  group  was  the  only  group  which  had  significant  differences  between  pre 
and  posttests  on  both  near  (p  <.01)  and  far  transfer  (p  <  .05).  The  closest  to  significance  in 
pre  and  posttests  on  far  transfer  was  found  within  cognitive  group 

{p  ~  .07).  Affective  and  control  groups  did  not.  have  significant  differences  between  pre  and 
posttests  on  both  near  and  far  transfer. 
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Table  3:         The  differences  between  post-pre  mean  scores,  standard  deviations, 
T  scores,  and  p  values  on  near  and  fa*  transfer  within  groups 


Groups 

Means  (post-pre) 

Std  Dev. 

T  Score 

P-values 

Affective- 

awareness 

Near 

.928 

3.15 

1.35 

.192 

Far 

-.095 

1.94 

-.0224 

.824 

Cognitive 

Near 

1.025 

3.53 

1.29 

.210 

Far 

.65 

1.56 

1.85 

.07 

Control 

Near 

.260 

4.77 

.262 

.79 

Far 

-.260 

1.91 

-.654 

.51 

Metacog. 

Near 

2.125 

3.30 

2.87 

.0096 

Far 

1.4 

2.32 

2.69 

.0144 

Qualitative  Findings 

Immediate  post  task  interviews,  on-line  data  collection,  and  student  task  design 
analysis  were  used  to  answer  the  two  qualitative  research  questions:  1)  how  subjects 
responded  to  metacognitive,  cognitive  and  affective  cues,  and  2)  how  metacognitive,  cognitive 
and  affective  cues  influenced  the  students'  task  designs  in  the  hypermedia  system  during 
the  training. 

To  answer  the  first  question,  coding  consisted  of  classifying  students'  responses  to  each 
question  cue  into  one  of  the  3  levels  of  focus.  The  following  is  a  list  of  the  categories  used  in 
the  coding  of  students'  responses  to  the  different  question  cues  embedded  in  the 
hypermedia-based  simulation  program: 

A.  Process-Oriented  Responses: 

1.  Planning  (one  step  ahead,  two  steps  ahead,  etc.) 

2.  Monitoring  (recognize  right/wrong  choices  and  why) 

3.  Subgoaling  ("I  decided  to  test  that  after  I  finished  this  because...") 

4.  Evaluative  ("This  experiment  seems  to  be  working  because...") 

5.  Modifying  ("I  realize  that  I  need  to  replicate  what  I  have  got  in  the  first  trial  to  draw 

more  reliable  conclusions.") 

B:  Cogni ti vc;- Ori e nted: 

1 .  Stating  specific  rules  for  the  problem  ("the  experiment  needs  to  control  variables,  etc.") 

2.  Stating  goals  of  the  problem  ("test  how  light  and  moisture  affect  isopod  behaviors") 

3.  Recognizing  the  current  state  of  the  problem  solving  ("  I  got  this  answer  right";  "my 
second  experiment  yielded  right  answer") 

C:   Personal  Feeling-Oriented: 

1 .  Positive  ("f  feel  OK,  fine,  confident,  etc/') 

2.  Negative  ("I  feel  frustrated":  "It  was  too  confusing") 

To  look  jit  how  subjects  in  the  different  treatment  groups  were  responding  to  the 
questions  that  were  asked  in  the1  program,  a  random  sample  of  6  protocols  from  each  of  the 
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treatment  groups  was  coded  for  the  content  by  two  independent  raters  according  to  the 
categories  classified  above.  The  summary  consisted  of  the  number  of  responses  judged  to  be 
a  member  of  each  category  for  each  question  cue  by  each  subject.  A  grand  summary  for  each 
treatment  was  then  calculated  by  each  rater.  The  interrater  reliability  indicated  90% 
agreement.  The  differences  were  resolved  by  discussing  the  responses  in  question  and 
deciding  upon  agreed-on  judgments. 

The  total  percentage  of  the  responses  made  in  each  major  category  by  each  group  was 
then  calculated  and  compared  to  find  out  whether  any  of  the  groups  differed  in  terms  of  the 
content  of  their  responses.  It  appears  that  each  group  focused  on  the  appropriate  level  of 
information  requested  by  the  questions  (see  Table  4).  The  affective  subjects  gave  96.5% 
feeling-oriented  statements,  3.5%  process  -oriented  statements  and  almost  no  cognitive- 
oriented  statements.  Among  feeling-oriented  statements,  65%  were  rated  as  positive  and 
35%  of  them  were  negative  statements.  Cognitive  subjects  gave  responses  that  fell  into  the 
problem-oriented  category  (rules,  goals,  and  problem  state)  85.7%  of  the  time.  14.3%  of  the 
time  they  made  process-oriented  responses  and  they  almost  made  no  feelings-oriented 
statements.  The  metacognitive  subjects  gave  72%  of  the  responses  that  fell  into  the  process 
category,  21%  of  the  responses  fell  into  problem  or  cognitive  level  and  7%  fell  into  feeling 
levels. 


Table  4:         Category  percentages  for  each  group 

Affective  Cognitive  Metacognitive 


Process  3.5%  14.3%  72% 

Cognitive  0  85.7%  21% 

Personal  95.5%  0  7% 

Total  100%  100%  100% 


In  further  analysis  of  the  interview  data,  it  was  found  that  metacognitive  subjects  were 
involved  more  in  the  reflective  processes.  When  they  reached  the  solution,  they  were  not  just 
satisfied  with  it,  but  went  further  to  find  out  why  it  was  right  or  wrong.  For  example,  when 
asked  to  summarize  how  the  questions  affected  the  whole  process  ,  one  student  responded 
"I  think  that  they  really  made  me  go  back  and  look  at  it  and  check  myself  to  see  that  I  was 
doing  them  right  and  really  looked  at  my  experiments  to  see  how  the  results  were  turned  out 
.Otherwise,  I  might  just  finish  and  not  go  over  it"  (emphasis  is  the  authors' ).  Another 
student  said  that  these  questions  helped  her  to  "  question  further  about  [her]  conclusions, 
like  why.  They  made  [her]  make  sure  that  they  were  valid  enough".  When  this  same 
student  was  asked  if  she  would  do  anything  differently  if  she  were  not  asked  the  questions, 
she  responded  "  Not  really.  I  probably  would  not  understand  the  processes  I  went  through 
this  well.  I  would  just  be  satisfied  that  I  got  them  right"  In  fact,  most  people  from  the  other 
3  groups  were  just  happy  to  get  the  right  answers  and  stopped  there.  One  student  in  the 
cognitive  group  summed  this  up  well .  She  said:  "  I  did  not  really  ask  myself  anything  at  the 
end.  I  was  just  glad.  I  knew  I  was  correct." 

Additionally,  it  appeared  that  while  solving  far  transfer  problems,  metacognitive 
subjects  continued  to  ask  themselves  questions  similar  to  those  presented  in  the  computer 
program.  This  was  not  indicated  by  other  groups  of  subjects.  One  student  in  the 
metacognitive  group  stated  "Yes.  I  did  go  through  the  similar  processes.  1  asked  myself  at 


the  first  place  why  I  knew  my  results  and  setting  up  tl.»s  experiments  are  right,  why  was  I 
confident  that  I  got  a  certain  part  right  because  I  remembered  my  computer  experiences". 
This  response  is  characteristic  of  11  of  the  14  metacognitive  students  interviewed. 

Affective  subjects  reported  that  awareness  of  on-task  feelings  helped  them  realize  how 
they  were  doing  in  the  problem  solving  process.  If  they  were  aware  that  they  were 
frustrated  or  nervous,  they  felt  that  there  must  be  something  wrong  with  their  performance. 
Rut  many  of  them  reported  that  they  did  not  know  what  was  wrong  or  how  to  fix  it.  ("It 
made  me  think  about  how  I  was  solving  the  problem  by  realizing  how  I  was  feeling  at  that 
moment.  Rut  the  question  did  not  help  me  solve  the  problem  better.  Because  they  did  not 
help  ine  find  out  where  is  my  problem  and  how  to  solve  it.")  Their  responses  told  us  that 
the  awareness  of  emotion  state  is  not  sufficient  to  enhance  the  problem  solving  performance. 

During  the  interview  of  control  subjects,  most  of  them  indicated  that  they  were  very 
confused  all  the  time  and  they  needed  more  help.  They  felt  that  decision-making  was  the 
toughest  part  in  the  whole  process.  They  were  glad  that  feedback  was  given  at  the  end  to 
help  them  realize  that  they  got  right  answers.  "I  feel  very  confused  the  whole  way  through.  1 
wish  I  had  more  help.  I  was  nc  Hire  if  I  was  right  or  wrong.  The  decision -making  part  was 
very  hard  on  me."  Overall,  subjects  across  groups  reported  increased  interest  and  confidence 
in  using  computers.  Many  of  them  said  "I  like  computer  experiments.  It  saves  time  and 
mess. (metacognitive  group)...";  "I  like  computers  and  I  would  feel  move  confident  if  I  have  to 
use  it  again. .(Cognitive  group).";  "I  think  that  it  was  very  neat  that  1  could  visually  see  what 
was  happening  in  my  experiments.  I  become  more  confident  in  using  computers  after  this 
hard  (control  group)";  and  "I  like  computers  and  wish  to  have  more  experience  with  it. 
(affective  group)". 

To  answer  the  question  of  how  metacognitive,  cognitive  and  affective  cues  influence 
students'  task  design  in  the  hypermedia  system  during  the  training,  a  constant  comparison 
analysis  was  performed  .  The  students  were  asked  to  diagram  their  designs  on  paper 
immediately  after  finishing  the  computer  program.  Figure4  1  shows  a  visual  representation 
of  the  process  of  forming  the  groups.  In  the  end,  the  designs  fell  into  one  of  four  (4)  large 
groups:  designs  exactly  like  those  on  the  program;  designs  which  did  not  show  any 
separation  of  variables;  designs  which  showed  a  separation  of  variables,  but  did  not  test  the 
two  variables  against  each  other;  and  designs  which  tested  the  variables  against  each  other. 
The  group  which  showed  separation  of  variables  was  subdivided  by  number  of  variables 
tested  (  1,  2  or  3)  and  whether  or  not  those  in  the  2  variable  category  ran  replications  or  not. 
This  resulted  in  four  subcategories  in  this  group.  The  group  which  showed  some  testing  of 
the  variables  against  each  other  was  also  subdivided  into  those  who  only  compared  the  two 
variables,  and  those  who  both  isolated  the  variables  and  compared  them.  This  last  group 
was  subdivided  further  as  to  whether  the  testing  of  the  variables  against  each  other  was 
correctly  designed  or  not,  and  if  the  correctly  designed  one  ran  replications  or  not.  This  gave 
a  total  of  four  subgroups  in  this  category,  making  a  total  often  possible  groups  of  designs 

In  each  of  these  ten  groups,  the  students  were  clumped  as  to  which  of  the  four 
experimental  groups  they  participated  in.  Numbers  of  students  in  each  group  were 
converted  to  the  percentage  of  the  total  number  of  students  from  their  treatment  group,  so 
comparisons  could  be  made  across  treatment  groups.  See  Table  5  for  these  results. 

The  ideal  task  design  would  be  to  both  isolate  variables  and  correctly  tested  them 
against  each  other(  group  10)  .  If  we  do  not  consider  whether  the  student  carried  out 
replications,  then  19%  of  the  cognitive  students  are  seen  in  this  group,  6.60%  of  the  affective 
students,  4,7%  of  the  control  students  and  4.5  %  of  the  metacognitive  students. 
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FIGURE1: 

A  Visual  Representation  of  the  Resulting  Groups  for  Analysis  of  Student-reported  Designs 


(Samne  As  Paula's  Design  ) 


(jDiffexent  f  som  Pauls' s  design  } 
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f  Combined  cowtctly J  (Combined  Wron 
(  Replication        j    (  No  replication  ) 


Table  5:  Comparison  of  Group  Compositions  for  Design  Analysis: 


group 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

control 

4.70 

4.70 

9.50 

9.50 

47.00 

0.00 

9.50 

4.70 

4.70 

0.00 

affective 

0.00 

13.30 

13.30 

6.60 

33.0 

13.30 

13.00 

0.00 

6.60 

0.00 

cognitive 

4.70 

4.70 

19.04 

19.04 

28.50 

0.00 

0.00 

0.00 

9.50 

9.50 

metacog 

9.00 

4.50 

4.50 

27.00 

32.00 

4.50 

0.00 

4.50 

4.50 

0.00 

Note:  group  numbers  in  this   able  refer  to  group  numbers  designated  in  Figure  1 


Conclusions 

On  the  posttest  of  far  transfer,  the  metacognitive  group  scored  significantly  better  (p 
<.001)  than  any  of  the  other  groups.  In  the  within-group  t  test  analysis,  the  metacognitive 
group  was  the  only  group  which  showed  statistically  significant  increases  from  the  pretest  to 
the  posttest  on  both  near  and  far  transfer  tests.  This  shows  that  the  metacognitive 
question  cues  can  positively  affect  the  way  in  which  students  approach  problems,  and  that 
this  effect  continues  afterward  for  problems  which  are  only  distantly  related  to  the  training 
problem. 
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More  evidence  for  this  carryover  of  thought  processes  is  revealed  in  the  student 
interviews.  When  asked  about  the  transfer  tasks,  students  in  the  metacognitive  group 
replied  that  they  continued  to  ask  themselves  questions  similar  to  the  ones  in  the  computer 
program,  while  the  other  cued  groups  did  not.  One  control  student,  when  asked  about  the 
post  test  answered  "  For  the  most  part  I  knew  I  was  right,  so  I  did  not  question  myself." 
Another  student  in  the  affective  group,  when  asked  if  she  asked  herself  anything  similar  to 
what  she  was  asked  on  the  computer  program,  replied  "No.  Not  really.  I  did  not  really  ask 
myself  anything  like  that."  During  the  interviews,  the  metacognitive  students  also  spoke  of 
the  fact  that  if  the  cues  had  not  been  there,  they  would  not  have  stopped  to  plan  as  well  as 
they  did,  nor  would  they  have  stopped  to  contemplate  why  they  got  correct  or  incorrect 
answers.  For  example,  one  student  when  asked  if  she  would  do  the  same  thing  if  she  had 
not  been  cued  replied  "No.  I  wouldn't.  I  may  just  be  satisfied  with  the  fact  that  I  got  it." 
The  same  student,  when  asked  if  the  questions  affected  her  confidence,  interest  or 
motivations,  said  ,rYes.  They  made  me  feel  good  because  1  got  them  all  right  and  I  did  the 
experiments  right  and  understand  the  whole  process  better..."  She  stated  that  if  she  had 
not  been  asked  the  questions  by  the  computer,  she  would  not  have  felt  the  same  way, 
''because  those  questions  helped  [her]  to  understand." 

There  was  variation  within  the  groups,  however,  and  some  students  replied  that  even 
though  they  were  forced  to  give  some  reply  to  the  cues,  they  did  not  give  much  thought  to  the 
cues.  One  student  commented,  "basically  I  just  used  pre-knowledge  from  the  lab  we 
originally  did,  the  hands-on  experiment.  I  used  prior  knowledge  to  decide  what  I  was  going 
to  do  next  as  I  had  originally  done."  One  student's  answers  indicated  that  although  she 
said  she  ignored  the  cues,  she  may  have  been  using  them.  When  asked  if  she  had  ignored 
the  questions,  this  student  replied,  "Exactly.  Those  questions  did  not  help  me.  But  they  did 
provoke  me  to  think  what  I  would  do  next.  But  1  could  not  find  the  direct  relationship 
between  what  I  have  thought  and  my  understanding  of  the  whole  process"  (  emphasis  is 
authors').  All  of  the  preceding  quotes  wen?  from  the  students  in  the  metacognitive  group. 
Although  the  responses  varied  among  these  students,  the  majority  of  them  (11/14) 
interviewed  indicated  that  these  cues  caused  them  to  stop,  reflect  and  do  something 
different  than  they  would  have  without  them. 

Although  the  qualitative  analysis  of  students'  task  designs  seems  to  contradict  this 
"good  news"  by  showing  that,  on  average,  the  metacognitive  group  did  worse  on  their  task 
designs,  we  are  reluctant  to  give  these  data  heavy  weight  for  several  reasons.  First,  the 
designs  were  self  reported  by  the  students  after  they  completed  the  computer  activity. 
Within  the  "recorded"  subset  of  students,  when  we  compared  their  self-reports  of  their  task 
designs  with  what  actually  happened  while  they  were  on  the  computers,  50%  of  the 
students  either  under  or  over  represented  the  training  task  designs  which  they  had  actually 
completed.  Second,  after  watching  the  screen  recordings,  we  found  that  many  of  those 
students  who  were  recorded  also  designed  tasks  which  were  not  actually  run,  or  changed  the 
design  of  the  task  before  they  ran  it.  The  students'  self-reported  data  were  not  complete. 
The  data  tended  to  show  only  a  segment  of  the  task  designs  and  did  not  accurately  show 
the  changes  that  students  made  during  the  process  of  designing  the  task.  In  analyzing 
these  data,  it  is  often  not  possible  to  tell  if  the  students*  diagrams  represent  an  initial  plan 
or  the  end  result  of  the  entire  process  of  computer  task  design.   It  can  be  concluded  from 
this  experiment  that  retrospective  qualitative  data  are  not  as  accurate  as  on-line  process 
data,  and  are  not  appropriate  for  collecting  process  data,  especially  when  the  data  can  not 
show  the  progress  or  changes  made  during  the  process. 

Therefore,  even  if  the  metacognit  ive  group  task  designs  were  less  comprehensive  than 
those  of  tho  cognitive  and  control  groups,  we  feel  that  this  is  not  an  indication  that  all  of  the 
c  ues  failed  (since  the  metacognitive  group  went,  on  to  outperform  the  other  groups  in  the 
transfer  test),  but  may  mean  that  some  of  the  eues  failed.  One  cue  that  may  not  have  been 
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successful  for  the  metacognitive  students*  task  design  was  the  first  cue  ("What's  your  plan 
for  solving  the  problem?").  This  cue  asked  the  students  to  detail  their  plans  for  the 
experiment. 

All  of  the  students  came  to  the  experiment  with  strong  preconceptions  carried  over  from 
the  experiences  in  the  hands-on  lab.  Unless  an  event  occurred  during  the  process  of  the 
computer  simulation  to  jar  them  out  of  this  preconception,  they  continued  to  solve  this 
problem  as  they  did  in  the  lab.  For  some  students,  reaching  wrong  answers  or  unexpected 
cues  made  them  confront  their  preconceptions.  Since  almost  60%  of  the  metacognitive  group 
ended  up  in  one  of  the  two  groups  which  only  tested  two  variables  independently  (see  Table 
5:  5&6),  it  may  be  that  the  cue  which  asked  them  "What's  your  plan  for  solving  the  problem?" 
was  not  different  enough  from  what  students  were  thinking  at  that  moment  to  elicit  a  strong 
response  and  thus  break  them  out  of  their  preconceptions. 

This  hypothesis  is  further  supported  by  the  interview  data.  When  asked  how  they 
reacted  to  the  first  cue,  most  of  the  metacognitive  students  indicated  that  the  main  effect  this 
cue  had  was  to  cause  them  to  stop  and  think,  or  to  slow  down,  For  example,  one  student 
responded,  "it  made  me  think  about  it  for  a  second  before  I  took  any  actions.  During  that 
second  it  reminded  me  of  what  I  did  in  class."  However,  while  doing  the  hands-on  lab 
experiment,  the  students  were  never  required  to  test  variables  against  each  other.  The 
problem -solving  process  they  went  through  in  the  hands-on  lab  could  not  be  applied  to  the 
problems  posed  in  the  computer  program.  Thus,  the  student's  response  was  not  appropriate. 
When  asked  if  they  would  have  done  anything  differently  if  they  had  not  been  given  this  cue, 
most  of  the  metacognitive  students  answered  that  the  only  difference  was  that  they  would 
not  have  written  down  their  plan,  or  that  their  plan  might  not  have  been  so  detailed. 
Another  student  responded  "I  put  'plan  to  test  variable  individually1.  I  did  not  make  a  very 
detailed  plan."  In  all  of  these  cases,  we  see  that  the  first  cue  did  not  succeed  in  breaking 
these  students'  preconceptions,  but  allowed  them  to  continue  their  task  designs  with  their 
preconceptions  intact.  This  might  explain  why  so  many  of  the  metacognitive  group  had  task 
designs  which  only  tested  the  two  variables  flight  and  moisture)  independently,  and  why  they 
did  not  go  on  to  test  the  two  variables  against  each  other. 

Contrasted  to  the  metacognitive  students'  responses  to  the  first  cues,  the  cognitive 
students'  interviews  showed  that  in  order  to  answer  their  first  cue  ("What  variable(s)  are 
you  testing  for?")  they  had  to  do  a  lot  of  metacognitive  thought.  For  example,  one  student 
said,  "They  really  made  me  think.  They  really  did.  They  helped  me  along  to  see  what  I 
need  to  do  in  an  experiment.  They  kind  of  making  me  think  about  exactly  what  variables  I 
need  to  test  and  how  many  I  am  supposed  to  have  in  the  experiment."  Another  student 
said,  "  They  really  made  me  think  how  to  set  up  certain  things.  And  it  just  made  me  think 
more  consciously  about  what  I  was  doing,"  This  tells  us  that  although  the  responses  to  this 
cue  were  cognitive 

(light, moisture,  etc.),  students  had  to  go  through  a  metacognitive  process  in  order  to 
respond  to  this  cue  appropriately.  In  fact,  within  the  cognitive  group,  9  of  the  12  interviews 
indicated  that  the  cognitive  students  went  through  such  metacognitive-oriented  processes 
when  responding  to  the  first  cue. 

We  can  conclude  that  a  cue  which  induces  a  metacognitive  process  can  have  a  significant 
positive  impact  on  students'  problem  solving  processes.  In  the  case  of  the  First  cue,  the 
metacognitive  response  was  actually  found  in  the  cognitive  group.  Consequently,  their  task 
designs  were  also  much  better  than  all  the  other  groups.  However,  this  single  cue  was  not 
strong  enough  to  guarantee  that  the  rest  of  their  thought  processes  continued  to  be 
metacognitively  oriented,  and  the  cognitive  students  failed  to  show  far  transfer. 

Even  though  the  first  cue  in  the  metacognitive  group  failed  to  induce  metacognitive 
processes,  the  metacognitive  students  subsequently  received  many  more  cues  which  actually 
induced  metacognitive  processes  (refer  to  Table  4  for  the  category  percentages  for  each 


group).  The  high  percentage  of  process  responses  generated  by  the  metacognitive  subjects 
suggest  that  the  majority  of  the  metacognitive  cues  successfully  engaged  students  in 
metacognitivc  processing.  This  allowed  the  students  to  continue1  a  process  of  "self- 
questioning"  during  the  posttest,  which  may  have  helped  them  score  significantly  higher  on 
the  posttest  of  far  transfer.  Therefore,  what  students  actually  transferred  were  the 
metacognitive  processes  rather  then  the  content- specific  information.  These  processes  made 
students  focus  on  and  become  consciously  aware  of  the  solution  components.  By  focusing  on 
a  process  level,  subjects  were  able  to  attend  to  information  regarding  the  way  in  which  they 
solved  a  problem.  This  finding  is  consistent  with  what  Berardi-Oolleta  (1990)  found  in  her 
study.  She  concluded  that  the  process-focused  groups  are  not  only  transferring  knowledge  to 
the  new  task,  but  that  they  are  also  transferring  an  approach  to  the  new  problem. 

Since  all  groups  claimed  to  have  planned,  regardless  of  the  type  of  cues,  we  propose  that 
metacognitive  planning  needs  to  be  carefully  interpreted  by  the  researcher.  Mere  planning 
was  not  sufficient  to  insure  correct  and  complete  task  design.  In  order  for  task  design  to  be 
correct  and  complete,  these  "plans"  had  to  be  coupled  with  active  metacognitive  processes 
such  as  monitoring,  reflecting  and  evaluating. 

The  issue  of  how  these  cues  affect  the  students'  task  design  process  needs  to  be  further 
explored.  Given  our  experiences  here,  we  would  recommend  that  the  process  data  be 
collected  concurrently  and  be  as  detailed  as  possible.  We  would  also  caution  that 
researchers  and  teachers  need  to  be  careful  about  their  choices  of  cues,  ensuring  that  the 
cues  are  attention  focusing.and  strong  enough  to  break  previously  held  preconceptions,  as 
well  as  inducing  metacognitive  processes.  Finally,  the  emphasis  in  problem  solving  research 
has  placed  on  needs  to  shift  to  focus  on  human  problem  solvers  as  "memory  banks"  to 
humans  as  dynamic.active  processors  of  their  own  learning  experiences  that  are  continually 
acquiring  information  in  the  learning  process. 


REFERENCES 

Authorware  Professional  (1987).  Authorware  Inc.  Mineapolos,  MN. 

Berardi-Coletta,  B.  (1990).  The  Effects  of  Verbalization  On  Problem  Solving  and  Problem 
Solution  Transfer-  A  Metacognitive  Perspective.  Unpublished  dissertation, 
University  of  Illinois  at  Chicago. 

Berry,  D.  C,  &  Broadbent,  D.  E.  (1987).  Explanation  and  verbalization  in  a  computer-assisted 
search  task.  The  Quarterly  Journal  of  Experimental  Psychology.  39A.  585-609. 

Borkowski,  J.  G.,  Carr,  M.,  Pellinger,  E.  &  Pressley,  M.  (1990).  Self-regulated  cognition: 

Interdependence    of  metacognition,  attributions,  and  self-esteem.  In  B.  F.  Jones  & 
Idol,  L.  (Eds.),  Dimensions  of  Thinking  and  Cognitive  Instruction.  NJ:  Erlbaum. 

Bourne,  D.  E.  (1990).  Computer-assisted  instruction,  learning  theory,  and  hypermedia:  An 
associative  linkage.  Research  Strategies.  8(4).  1KQ-171. 

Brown,  A.  L.,  Bransford,  J.  D.,  Ferrara,  R.  A.,  &  Campione,  J.  C.  (1983).  Learning,  remembering, 
and  understanding  In  J.  H.  Flavell  &  E.  H..  Markman  (Eds.),  Handbook  of  Child 
Psychology;  Cognitive  Development,  (Vol.  3)  New  York:  Wiley. 


478 


ERIC 


ATI 


Butte  rfield,  E.  C,  &  Nelson,  G.  (1991).  Promoting  positive  transfer  of  different  types.  Cognition  And 
Instruction,         SQL  69- 102, 

Clark,  R.  E.  (1992).  How  the  cognitive  sciences  are  shaping  the  profession.  In  Stolovitch,  H. 
D.,  &  Keeps,  E.  J.  (Eds).  Handbook  of  Human  Performance  Technology:  A 
Comprehensive  Guide  for  Analyzing  and         Solving  Performance  Problems  in 
Organizations.  New  York:  Maxwell  Macmiilan. 

Corno,  L.  (1986).  The  metacognitive  control  components  of  self-regulated  learning. 
Contemporary  Educational  Psychology.  1 1,  333-346. 

Corno,  L.;  &  Mandinach,  E.  B.  (1983).  The  role  of  cognitive  engagement  in  classroom  learning  and 
motivation.  Educational  Psychologist.  18.  88-108. 

Dominowski,  R.L.  (1990).  Problem  solving  and  metacognition.  in  K.  J.  Gilhooiy,  M.  T.  G. 

Keane,  R.  H.  Logie,  and  G.  Erdos  (Eds.)  Lint's  of  Thinking.  Volume  2.  John  Wiley  & 
Sons  Ltd. 

Flavell,  J.H.  (1976).  Metacognitive  aspects  of  problem  solving.  In  Resnick,  L.  0  The  Nature  of 
Intelligence.  Hillsdale,  N.  J.:  LEA. 

Hagnc.  N.  L.f  &  Berliner,  D.  C.  (1988).  Educational  Psychology.  Boston:  Houghton  Mifflin 
Company,  (pp.  311-327). 

(iagne.R.  M.t  &  Smith,  E.  C,  (1962).  A  study  of  the  effects  of  verhalization  on  problem  solving. 
Journal  of  Experimental  Psychology.  63(1).  12-18. 

(■ick,  M.  L,  &  Uolyoak,  K.  J.  (1980).  Analogical  problem  solving.  Cognitive  Psychology.  12.  306- 
335. 

Glaser,  R.  (1984).  Education  and  thinking:  The  role  of  knowledge.  American  Psychologist. 
29,  93-104. 

1  Anderson,  II.  W.  (1986).  Self-regulated  learning:  Implications  for  the  design  of  instructional 
modules.  Contemporary  Educational  Psychology.  U.  405-427. 

Linn,  M.  C.  (1983).  Content,  context  and  process  in  adolescent  reasoning.  Journal  of  Early 
Adolescence.  3.  63-82. 

McCombs,  B.L.  &  Marzano,  R.  J.  (1990).  Putting  the  self  in  self-regulated  learning:  The  self 
as  agent  in  integrating  will  and  skill.  Educational  Psychologist.  25(1).  51-69. 

Paske,  R.  (1990).  Hypermedia:  A  brief  history  and  progress  report.  T.  H.  E.  Journal.  Aug. 

53-56. 

Pintrich,  P.  R..  Cross,  D.  R.,  Kozma,  R.  B.,  &  McKcachie,  W.  J.  (1986).  Instructional 
psychology.  Annual  Review  of  Psychology.  £L  611-651. 


479 


Pint  rich,  P.  It.  &  De  Groot,  E.  V.  (1990).  Motivational  and  self-regulated  learning 

components  of  classroom  academic  performance.  Journal  of  Educational  Psychology. 

aam  33-40. 

Rowe,  11.  (1988).  Metacognitive  skills:  Promises  and  problems.  Australian  Journal  of 
Reading.  11(4).  227-237. 

Salomon,  (J.  (1988).  AJ  in  reverse:  Computer  tools  that  turn  cognitive.  Journal  of 
Educational     Computing  Research,  4  (2).  123-134. 

Salomon,  D.,  &  Perkins,  D.  N.  (1989).  Problem-solving  strategies.  Educational  Psychologist. 
iL_113-142. 

Spiro,  It.,  &  Johng,  J.  (1990).  Cognitive  flexibility  and  hypertext:  Theory  and  technology  for 
I  he  nonlinear  and  multidimensional  traversal  of  complex  subject  matter.  In  1).  Nix 
&  R.  Spiro  (Eds.),  Cognition.  Education  and  Media  (pp.  163-20B).  Hillsdale,  NJ: 
Lawrence  Erlbaum  Associates. 

Stinessen,  L.  (1985).  The  influence  of  verbalization  on  problem  solving.  Scandinavian 
Jwrnal  of  Psychology.  26.  342-347. 

Thomas,  R.  H.  Jr.  (1974).  An  analysis  of  behavior  in  the  hobbits  orcs  problem.  Cognitive 


Zimmerman,  B.  J.  (1990).  Self-regulated  learning  and  academic  achievement:  An  overview. 
Educational  Psychologist.  25(1),  3-17. 

Zimmerman,  B.  J.  (1986).  Becoming  a  self-regulated  learner:  Which  are  the  key  subprocesses? 
Contemporary  Educational  Psychology.  11-  307-313. 


480 


<'i'7D 


APPENDIX  l:Cues  Embedded  Within  the  Computer  Program 


Affective  Group 

Cognitive  Group 

Metacognitive  Group 

Location  One:          after  reviewing  Paula's  experiment  and  problem  statements,  and 

before  any  selections  have  been  msde 

Rationale:  check  feelings 
and  mood  before  task 

Rationale:  review  the  goals 
and  rules  of  solving  the 
problem 

Rationale:  make  plans  for 
solving  the  problem 

(1)  How  are  you  feeling  right 
now  in  dealing  with  this 
problem? 

(1)  What  variables  are  you 
testing? 

(1)  What  is  your  plan  for 
solving  the  problem? 

Location  Two:       after  selecting  all  of  the  materials  and  supplies,  and  before 

conducting  the  experiments 

Rationale:  check  on-task 
feelings 

Rationale:  check  actions 
against  goals  and  rules  of 
solving  the  problem 

Rationale:reflect  on  the 
actions  have  been  taken  and 
plan  for  the  next  step 

(2)  How  are  you  feeling 
right  now? 

(2)  What  materials  and 
supplies  have  you  selected? 

(3)  What  are  you  going  to 
test? 

(2)  Why  did  you  select 
those  materials  and 
supplies? 

(3)  How  are  you  deciding 
what  to  do  next? 

Location  Three:       after  finishing  all  of  the  experiments  and  drawing 

conclusions,  and  before  the  feedback  was  given 

Rationaleicheck  post-task 
feelings 

Rationale:encourage  self- 
discovery  of  the  current  sate 
of  the  problem  solving 

Rationale:  monitor  and 
evaluate  the  effectiveness  of 
the  process  of  the  problem 
solving 

(3)  How  you  feeling  right 
now? 

(4)  What  conclusions  can  you 
draw  about  the  variable(s)? 

(4)  How  did  you  decide  that 
you  have  had  enough  data  to 
make  conclusions? 

Location  Four: 

after  providing  the  feedback 

Rationale:check  the 
processes  of  the  feeling 
change 

Ration alercheck  the  final 
performance  by  knowing  the 
number  of  the  experiments 
that  are  useful  or  not  useful 

Rationalerdo  not  be 
satisfied  with  right  or  wrong 
answers,  find  out  why  they 
are  right  or  wrong,  build  up 
a  repertoire  of  strategies  for 
further  modifications  of  the 
processes 

(4)  How  are  you  feeling  right 
now  comparing  to  when  you 
just  got  started? 

(5)  How  many  of  your 
experimental  designs  gave 
useful  data?  or 

How  many  of  your 
experimental  designs  did  not 
ftive  useful  data? 

(5)  What  part  of  your 
experiments  led  you  to  the 
right  or  wrong  conclusions? 
*How  are  you  going  to  fix 
it?(ask  only  if  the  conclusions 
are  wrong.) 
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Appendix  2:  Teacher-Made  Near  and  Far  Test  of  Problem  Solving 


NAME   DIVISION  

BIOLOGY  205  LAB  QUIZ      FALL  1992 

INDIVIDUAL  QUIZ  -  ISOPOD  LAB 

1.  What  environmental  conditions  do  isopods  "prefer?"  (4  pts.) 

2.  Explain  the  basis  for  your  conclusions  about  isopods1  "preferred"  environmental 

conditions.  (6  pts.) 

3.  Suppose  you  had  the  following  information  for  the  isopod  lab: 

Ten  isopods  were  placed  in  a  trough  in  section  3,  and  after  ten  minutes,  the  following 
results  were  noted  as  diagrammed  below: 
WET  DRY 


* 

★ 

* 
★  ★ 
* 

★ 

1 

2 

3 

4 

6 

Based  on  the  information  given  above,  answer  the  following  questions: 

a.  What  is  the  variable  in  this  experiment?  (1  pt.) 

b.  How  would  you  describe  the  response  of  these  organisms  to  this  variable  (based  on 
the  diagram        above)?  (2  pts.) 

c.  What  is  random  behavior?  (1  pt.) 

d.  Do  you  see  random  behavior  in  the  diagram  above?  Explain  your  answer.  (3  pts.) 

e.  What  could  you  do  to  validate  your  results?  (3  pts.) 


4.  QUESTION  (6  POINTS  TOTAL)  (FAR  TRANSFER  PROBLEM) 

The  elementary  school  in  which  you  teach  seems  to  have  an  unusually  high  number  of  hyperactive 
students.  In  fact,  of  the  100  students  in  second  grade,  50  of  them  are  hyperactive.  Different 
groups  of  adults  have  proposed  various  ideas  to  explain  this.  The  parents  say  that  the  peer 
influence  causes  them  to  be  hyperactive.  The  principal  thinks  that  the  family  environment  is 
the  cause  of  their  hyperactivity.  The  teachers,  on  the  other  hand,  say  that  diet  is  the  source  of  the 
condition.  You  have  been  asked  to  design  an  experiment  that  would  determine  which  one  or  more 
than  one  of  these  explanations  is  correct.  Explain  how  you  would  set  up  this  experiment  to 
determine  the  cause  or  causes  of  the  students'  hyperactivity.  (Use  the  back  of  this  sheet  if  you 
need  more  space.) 
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ABSTRACT 


In  learning  environments  which  provide  little  structure  (e.g.,  hypermedia  systems), 
students  have  an  especially  strong  need  to  regulate  and  control  their  own  learning.  Such 
learning  environments  should  be  ideal  for  enhanced  learning.  However,  a  review  of  the 
literature  indicates  that  merely  providing  learners  with  flexible  learning  environments  does 
not  necessarily  mean  that  they  will  effectively  explore  and  learn  meaningfully  from  the 
resources  provided.  It  has  been  suggested  from  previous  research  that  poor  metacognitive 
skills,  that  is  the  inability  to  accurately  monitor,  reflect,  evaluate  and  adjust  learning, 
hinders  learning  in  this  free  environment.  Metacognitive  theory  and  research  are  reviewed 
and  recommendations  are  made  for  future  research  in  hypermedia-based  instruction  and 
learning. 
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Metacognition:  Implications  for  Research  In  Hypermedia-Based 
Learning  Environment 

Hypermedia  is  an  advanced  technology  that  combines  film,  video,  computer  graphics, 
sound,  music,  and  text  in  a  unified  information-delivery  system  centered  upon  the  personal 
computer  (Paske,  1990).  The  authoring  capability  of  popular  products  used  to  develop 
hypermedia-based  instruction  makes  the  development  and  production  processes  relatively 
easy  and  rapid.  Many  advantages  make  hypermedia  systems  attractive  delivery  alternatives 
for  both  school  education  and  industrial  training  purposes. 

The  use  of  hypertext  and  hypermedia  systems  as  an  effective  instructional  aid  "is 
premised  on  a  belief  in  learner  control"  (Jonassen,  1989;  p.  49).  For  example,  the  nonlinear 
nature  of  hypermedia  systems  requires  users  to  decide  when  and  where  to  enter  and  exit  the 
system,  how  much  information  is  needed,  what  kind  of  information  should  be  selected,  where 
to  find  it,  and  what  approach  is  the  best.  It  is  apparent  that  independent  decision  making,  as 
well  as  understanding  and  effectively  regulating  one's  own  learning  process,  is  crucial  for  an 
individual  to  successfully  exercise  the  control  required  by  hypermedia-based  environments. 

For  the  learner,  the  advantages  of  using  hyper-based  text  and  media  include  the  ability 
to  augment  learning  by  providing  a  means  to:  (a)  immediately  make  connections  among  a 
variety  of  knowledge  sources;  (b)  derive  potential  relationships  among  topics  of  information; 
and  (c)  interlock  webs  of  information  (Balajthy,  1990).  Additionally,  from  a  motivational 
viewpoint,  this  mode  of  presentation  can  greatly  increase  the  learner's  contro  ,  allowing  for 
an  emphasis  on  personally  relevant  topics  and/or  the  elimination  of  unwanted  topics  from 
review.  Increased  learner  control  may  also  lead  to  learning  achievements  being  more  readily 
attributable  to  the  individual,  thus  increasing  positive  perceptions  of  self-efficacy  (Schunk, 
1991). 

As  Jonassen  (1989)  points  out,  learner  control  is  an  instructional  strategy  that  is 
grounded  on  the  belief  that  the  learner  knows  what  is  best  for  him  or  herself  and  will  be 
more  motivated  toward  learning  if  given  control  of  learning  decisions.  However,  "the 
research  on  learner  control  has  not  been  supportive"  of  this  premise  (Jonassen,  1989;  pp.  49). 
It  has  repeatedly  been  shown  that  learners  do  not  make  the  best  decisions  when  given 
control  over  instruction  and  in  many  cases  actually  learn  less  than  those  not  given  such 
control  (Jonassen,  1989;  Williams,  1993). 

-  Several  potential  explanations  may  help  clarify  the  gap  between  the  potential  learning 
benefits  and  the  empirical  findings.  One  such  alternative  is  simply  that  learner  control  is 
not  an  effective  approach  to  learning  new  material.  Given  this  alternative,  it  would  be 
hypothesized  that  *  reduction  in  learner  control  with  a  reciprocal  increase  in  program  control 
would  lead  to  increased  learning.  Findings  from  studies  designed  to  test  this  hypotheses, 
however,  have  also  been  inconsistent.  Moreover,  imposing  conlrol  essentially  negates  the 
flexibility  and  potential  motivational  value  of  hypertext  and  hypermedia  technology. 

A  second  explanation  focuses  on  the  possibility  that  learner  control  does  make  a 
positive  difference  in  learning,  but  traditional  tests  of  this  hypothesis  have  been  inadequate. 
Both  Marchionini  (1988)  and  Jonassen  (1988)  indicated  that  new  evaluation  criteria  are 
needed  to  assess  knowledge  structures  and  student  learning  in  learner  controlled 
hypertext/hypermedia  environments.  Gleim  and  Harvey  (1992)  reviewed  and  discussed 
several  methodologies  that  may  permit  a  better  view  of  what  actually  is  occurring  within 
these  environn:  jnts. 

A  third  alternative  suggested  by  several  researchers,  and  the  one  that  is  the  focus  of  this 
paper,  is  that  learners"  lack  of  metacognitive  knowledge  and  skills  of  the  users  may  hinder 
their  learning  in  situations  of  increased  learner  control  (e.g.,  Balajthy;  1990;  Park,  1992; 
Steinberg,  1989;  Williams,  1993). 
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Metacognition  is  usually  defined  as  the  awareness  and  active  monitoring  of  one's 
cognitive  processes  (Brown,  1987,  Flavell,  1987).  Individuals  with  higher  levels  (or  more 
developed)  metacognitive  skills  tend  to  plan  cognitive  actions,  recognize  personal 
capabilities,  monitor  their  progress,  and  reflect  upon  their  own  mental  processes,  as  well  as 
regulate  those  processes  (Brown,  Bransford,  Ferrara  &  Campione,  1983).  Metacognitive 
skills  thus  help  individuals  learn  independently,  guide  their  own  learning  processes,  control 
the  selection  and  sequence  of  both  content  and  process,  and  transfer  what  has  been  learned 
to  other  situations  (Rowe,  1988;  Lin,  1993).  These  skills  enhance  the  individual's 
independent  thinking  and  learning-how* to- learn  abilities.  Because  hypermedia  systems  rely 
heavily  on  learners  to  make  decisions,  appropriate  inferences,  and  adjustments  to  the 
changing  learning  environment,  it  seems  plausible  that  a  fairly  high  degree  of  metacognitive 
knowledge  and  skills  would  be  important  for  successful  learning  within  such  environments. 

As  hypertext/hypermedia  becomes  increasingly  popular  in  education  and  training,  there 
is  a  greater  need  for  instructional  technologists  to  understand  the  construct  of  metacognition 
and  its  role  in  comprehension  and  learning.  Metacognitive  skills  appear  critical  to  effective 
use  and  learning  from  hypermedia  systems  and,  reciprocally,  hypermedia  formats  may 
provide  a  fertile  environment  for  the  learning  of  such  metacognitive  skills.  If  effective 
metacognitive  instructional  strategies  can  be  incorporated  into  hypermedia-based  learning 
environments,  they  may  in  fact  foster  the  development  of  good  thinkers  who  are  successful 
problem- solvers  and  lifelong  learners. 

The  purpose  of  this  paper  is  to  examine  research  on  metacognitive  theory  and  examine 
its  implications  for  hypermedia-based  instruction.  The  paper  (1)  discusses  the  promises  and 
limitations  for  learning  in  hypermedia  systems;  (2)  reviews  research  findings  from  the  field 
of  metacognition;  (3)  discusses  implications  of  the  findings  for  research  in  hypermedia 
learning  environments:  and  (4)  suggests  areas  for  future  research. 

Learning  In  Hypermedia  Environment 

The  most  frequently  cited  hypertext  system  capability  is  the  provision  of  high  levels  of 
flexibility  and  freedom  for  learner*  to  explore,  construct  meaning  and  generate  their  own 
learning  models  (McGrath,  1992).  For  example,  hypermedia  systems  provide  the  freedom  to 
navigate  through  the  system  in  whatever  manner  the  user  desires  (Kozma,  1991).  Such 
systems  also  provide  means  whereby  the  user  can  add  his  or  her  own  information  and 
construct  their  own  relationships  (Kozma  &  Van  Roekel,  1986). 

Learner  control  is  a  basic  premise  within  hypertext/hypermedia  environments  (Jonasson, 
1989).  It  is  left  up  to  the  learner,  based  on  his  or  her  goals  and  previous  experiences,  to 
decide  what  to  explore,  how,  and  when  to  explore  it  (Roselli,  1991).  Because  of  a  wide  variety 
in  those  past  experiences,  and  in  the  manner  in  which  individuals  learn,  it  has  been 
hypothesized  that  "...instruction  generally  increases  in  effectiveness,  efficiency,  and  appeal  to 
the  extent  that  it  permits  informed  learner-control  by  motivated  learners"  (Reigeluth  & 
Stein,  1983). 

Why  should  the  effectiveness,  efficiency,  and  appeal  of  instructional  materials  be 
enhanced  through  informed  learner  control0  \ccording  to  Williams'  (1993)  recent 
comprehensive  review  of  the  learner  control  «cerature,  instructional  effectiveness  (i.e.,  the 
degree  to  which  the  material  is  learned),  may  in  part  be  dependent  upon  the  levels  of 
invested  mental  effort  required  of  the  learner.  "One  might  expect  a  learner-controlled 
instructional  treatment  to  induce  more  elaborate  mental  processing  from  the  students  as  a 
result  of  their  pondering  the  choices  with  which  they  are  faced"  (p.  6).  Several  theorists 
argue  that  greater  mental  effort  increases  the  probability  of  meaningful  elaboration  resulting 
in  deeper  comprehension  (Salomon,  1983;  Stein  &  Bransford,  1979).  Additionally,  from  a 
constructivist  point  of  view,  providing  a  learning  context  in  which  the  learner  participates  in 


486 

4S5 


the  construction  of  mental  structures,  in  place  of  having  those  structures  predetermined, 
facilitates  learning  (Ertmer  &  Newby,  1993;  Hartley,  1985). 

From  an  efficiency  standpoint,  increased  levels  of  learner  control  allow  instruction  to  be 
tailored  by  the  individual  learner  so  that  it  is  relevant  to  his  or  her  own  goals  and  capacities 
(Roselli,  1991),  thus  limiting  or  eliminating  redundant  or  undesirable  learning  materials  and 
experiences. 

Finally,  the  appeal  of  instruction  and  its  motivational  value  should  also  be  affected  by 
the  degree  of  learner  control.  Steinberg  (198$  oointed  out  that  in  learner-controlled 
computer  environments,  a  learner's  boredom,  frustration,  and  anxiety  should  be  reduced. 
Within  such  environments,  when  faced  with  a  boring  task  the  learner  has  the  ability  to 
choose  to  alter  and  add  variety  to  what  is  presented.  This  variety  may  increase  the  level  of 
appeal  of  the  instruction  (Keller,  1987).  Similarly,  when  unfamiliarity  increases  feelings  of 
frustration  or  anxiety,  users  have  the  opportunity  to  postpone  exploration  of  the  system  to 
select  other  materials.  Additional  motivational  benefits  of  learner  control  include  possible 
elevated  levels  of  confidence  being  acquired  by  the  learner  (Keller,  1987),  increased  levels  of 
self-efficacy  (Schunk,  1991),  and  increased  levels  of  continuing  motivation  (Kinzie  & 
Sullivan,  1989).  In  each  of  these  cases,  the  benefits  are  based  on  the  learner's  ability  to 
attribute  success  more  readily  to  him  or  herself  because  of  the  control  over  the  learning 
experience. 

Although  many  potential  benefits  from  learner  control  have  been  suggested  for  several 
yours,  to  date,  the  research  does  not  support  its  unconditional  use  (Williams.  1993).  In 
reviewing  the  literature  on  learner  control  in  computer-based  instruction,  it  has  been  shown 
that  when  students  are  given  control  over  sequence,  path,  content,  pace,  etc.,  their 
performances  are  mixed  in  terms  of  effectiveness,  efficiency  and  appeal.  Frequently,  results 
■  it*  learner  control  studies  indicate  performance  at  levels  similar  to  or  significantly  lejyj  than 
students  who  are  not  given  controls  (Rubincam  &  Oliver,  1985;  Steinberg,  Baskin,  & 
Matthews,  1985;  Williams,  1993).  Rubincam  and  Oliver  (1985).  for  example,  surveyed  eleven 
studies  to  compare  students'  achievements  in  .earner-controlled  versus  computer-controlled 
situations.  Five  of  the  studies  showed  superior  performance  under  non-learner  controlled 
conditions  and  only  two  under  learnercontrolled  conditions.  The  rest  of  the  studies  indicated 
no  differences  between  the  two  conditions.  Researchers  in  the  area  have  found  that  users 
often  exit  lessons  too  soon,  view  too  few  examples,  and  complete  the  lesson  too  quickly 
(Tennyson,  Tennyson,  &  Rothcn,  1980;  Ross  &  Rakow.  1980). 

Rubincam  and  Oliver  (1985)  conducted  a  study  involving  a  high  school  CAI  algebra 
lesson.  Students  controlled  the  sequence  of  objectives  within  each  topic  and  decided  whether 
to  take  instruction  before  the  test  or  go  to  the  test  immediately.  It  was  found  that  students 
who  had  a  consistent  selection  strategy  and  a  reason  for  selecting  their  strategies  performed 
better  than  those  who  did  not.  However,  being  able  to  select  strategies  consistently  required 
learners  to  know  what  was  best  for  the*  selves  concerning  the  task  at  hand  and  the 
environments  in  which  they  were  studying.  Learners  had  to  be  able  to  select  effective 
learning  strategies,  and  to  understand  when  and  why  those  strategies  should  be  used.. 

In  hypermedia-based  learning  environments,  users  are  given  even  more  control  over  the 
instruction  than  in  traditional  CAI.  Moreover,  easy  access  fo  the  vast  quantities  of 
information  (Marchionini,  1988),  the  increased  number  of  learning  options  available  in  the 
systems  (Jonassen,  1989),  and  the  richness  of  the  non-linear  representation  of  information 
(Dede,  1988)  can  make  learning  in  hypermedia  systems  extremely  challenging  and  difficult. 
It  is  likely  to  be  especially  difficult  for  those  users  who  lack  skills  in  planning,  monitoring, 
reflecting,  and  regulating  their  personal  learning  strategies.  The  increased  complexity  and 
lack  of  structure  of  such  environments  automatically  impose  increased  responsibilities  and 
cognitive  processing  requirements  on  users.  In  these  circumstances,  it  is  important  for 
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individuals  to  be  able  to  select  and  generate  strategies  to  guide  their  own  learning  processes 
and  explore  thought  itself.  They  should  be  able  to  identify  (to  oneself  or  others)  how  a 
particular  strategy  came  into  being,  why  particular  options  were  chosen,  and  how  particular 
conclusions  were  reached.  These  metacognitive  skills  enable  users  to  control  their  own 
cognitive  processing  internally.  Lacking  these  abilities,  users  may  become  confused  and  "get 
lost"  in  hypermedia  systems.  They  may  not  be  able  to  discern  what  they  want,  where  they 
are  in  the  network,  or  how  to  reach  any  specific  point.  These  problems  are  known  as  learner 
cognitive  overload  and  disorientation  (Marchionini,  1988;  Roselli,  1991). 

Despite  the  importance  of  possessing  metacognitive  skills,  they  often  are  not  clearly 
demonstrated  by  young  children  or  even  college  students  (Wong,  1989).  In  addition,  the 
overall  quality  of  users'  metacognitive  skills  is  relatively  poor  (Schoenfeld,  1985).  It  seems 
that  the  problems  in  attaining  the  potential  benefits  of  learner-controlled  learning 
environments  may  be  attributable,  at  least  in  part,  to  the  lack  of  the  learner's  use  of 
metacognitive  knowledge  and  skills.  In  our  view,  such  environments  require  users  to  engage 
in  a  variety  of  metacognitive  activities,  including  being  aware  of  the  nature  of  personal 
learning  styles  and  deficiencies  and  planning,  monitoring,  and  decision  making  during 
learning. 

Providing  External  Support 

To  fulfill  the  learning  potential  in  hypermedia-based  learning  environments,  the  creation 
of  external  supports  as  a  supplement  to  users1  abilities  has  been  advocated.  One  external 
support  that  is  frequently  used  is  adaptive  advisement,  a  technique  in  which  the  computer  is 
used  to  diagnose  the  learner's  understanding,  prescribe  the  instructional  procedures,  explain 
underlying  principles  and  processes,  etc.  Another  common  support  is  to  use  graphics,  maps, 
charts,  etc.,  that  show  students  the  structure  of  the  knowledge  in  the  system.  Designers  of 
hypermedia  systems  have  also  used  metacognitive  aids  (on-line  ability  to  save  self-generated 
explanations,  highlighting  of  unfamiliar  terms,  etc.).  Overall,  students  provided  with 
external  supports  perform  significantly  better  in  immediate  tasks,  but  they  do  not  maintain 
or  transfer  their  knowledge  very  well  (Steinberg,  Baskin,  &  Matthews,  1985;  Steinberg, 
Baskin,  &  Hofer,  1986;  Tennyson  &  Rothen,  1979;  Tennyson,  1980).  In  fact,  these  external 
supports  may  themselves  be  burdens.  The  so-called  metacognitive  aids  provided  within  some 
hypermedia  systems  (e.g.  program-provided  explanations  of  the  process  one  has  gone 
through)  are  frequently  ignored  by  users  and  are  viewed  as  additional  cognitive  load  (Recur 
&  Pirolli,  1992).  Marchionini  (1988)  pointed  out  that  novices  use  external  help  to  avoid 
cognitive  overload  rather  than  as  a  way  of  dealing  with  it. 

These  findings  support  Bernard-Colleta's  (1990)  contention  that,  unless  learners  are 
directed  to  be  involved  in  the  metacognitive  process  themselves,  many  will  not  automatically 
engage  in  metacognitive  processes  (Brown,  Bransford,  Ferrara  &  Campione,  1983).  These 
results  suggest  that,  although  external  supports  can  help  students  solve  problems  initially, 
they  do  not  necessarily  help  learners  develop  their  own  strategies  and  other  required 
metacognitive  skills  (Steinberg,  1989).  Too  many  external  supports  may  lead  students  to 
ascribe  the  success  or  failure  of  their  learning  to  external  factors  and  subsequently  take  less 
responsibility  for  their  own  learning.  Their  motivations  to  develop  strategics  may  even  be 
severely  impaired  if  too  many  external  supports  are  provided  (Steinberg,  1989).  ThuH,  the 
mere  provision  of  an  external  support  is  not  necessarily  beneficial,  at  leant  for  long-term 
retention  and  transfer,  if  learners  do  not  und  erstand  how  it  can  help,  and  when,  whore  and 
why  to  use  it. 

Most  of  the  external  support  discussed  above  may  have  lacked  the  tuipport  niTclrd  to  hi»lp 
V  arners  develop  appropriate  internal  learning-how-to-learn  abilities  Learning  may  hot  he 
)  itimal  in  hypermedia  systems  because  users  have  used  inappropriate  learning  HtrutegicH 
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(Steinberg,  1977),  because  they  lack  the  knowledge  and  skills  or  metacognitive  skills  (Merrill, 
1980;  Rigney,  1978)  needed  to  make  meaningful  decisions  (Steinberg,  1989;  Tennyson  & 
Rothen,  1979)  for  a  combination  of  these  reasons.  In  reviewing  the  literature  on  hypermedia 
and  metacognition  for  disabled  readers,  Balajthy  (1990)  came  to  similar  conclusions.  He 
suggested  that  poor  metacognitive  functioning,  defined  as  students'  inability  to  monitor 
accurately  their  own  success  or  failure,  hindered  learning  in  situations  in  which  students 
were  given  control,  as  opposed  to  situations  in  which  control  was  exerted  by  the  computer. 
Therefore,  it  is  important  to  look  at  the  construct  of  metacognition  and  research  findings 
arising  from  metacognitive  training  to  suggest  how  to  enhance  the  effectiveness  of 
hypermedia- based  learning  environments. 

The  Need  for  Learner-Generated  Metacognitive  Processes 
Research  related  to  cognition  focuses  on  how  thoughts  and  internal  mental  processes 
influence  learning  and  behavior  as  well  as  how  these  processes  take  place  successfully  or 
unsuccessfully.  In  the  process  of  acquiring  knowledge,  a  learner  applies  various  "cognitive 
strategies."  For  example,  a  person  who  must  learn  to  discriminate  among  several  sets  of 
numbers  might  apply  a  categorizing  strategy  to  help  distinguish  one  set  from  the  others.  An 
instructor  might  employ  an  association  strategy  at  the  beginning  of  a  new  class  in  order  to 
memorize1  the  names  of  students..  Cognitive  abilities  can  thus  be  regarded  as  "enabling 
variable*"  in  that  thoy  facilitate  performance  of  specific  tasks  (Rowe,  1988,  p.  228). 

Theorists  who  distinguish  between  cognitive  and  metacognitive  processes  use  the  term 
metacognition  to  refer  to  those  aspects  of  cognition  that  require  learners  to  understand  their 
own  cognitive  processes  and  to  monitor,  direct  and  control  them.  Flavell,  who  pioneered  the 
field  of  metacognition  with  his  exploration  of  metamemory,  describes  it  this  way: 

Metiini^iiihun  is  usually  defined  as  knowledge  and  cognition  about  cognitive  objects, 
that  in,  about  anything  cognitive.  However  the  concept  could  reasonably  be  broadened  to 
include  anything  psychological,  rather  than  just  anything  cognitive.  For  instance,  if  one  has 
knowledge  or  uictacngnmve.  (Flavell,  1987,  p.21) 

Itmwn  CltWYI  def  ines  the  concept  of  metacognition  as  the  understanding  of  knowledge 
that  can  lie  i  elicited  in  cit  her  effective  use  or  overt  description  of  the  knowledge  in  question. 
Within  thi'  psychological  literature,  the  term  has  been  used  to  refer  to  two  distinct  areas  of 
research  knowledge  ahum  cognition  and  the  regulation  of  cognition  (Brown,  1987). 

Knowledge  About  Cognition 

For  Kluuell  (MIH7),  metacognitive  knowledge  consists  of  three  major  variables:  person 
variables,  task  variables  and  strategy  variables.  Knowledge  of  the  three  variables  may 
influence  a  person's  performance. 

Person  variables  involve  knowing  about  oneself  as  a  learner,  e.g.,  one's  cognitive 
strengths,  weaknesses,  abilities,  motivation,  attitudes,  etc.  In  hypermedia-based  learning 
environments,  this  knowledge  will  include  one's  beliefs  concerning  one's  own  ability  to 
interact  with  the  environment  as  well  as  one's  understanding  of  personal  affective  states, 
such  as  motivation,  interests,  curiosity  and  anxiety.  Personal  knowledge  can  also  include 
one's  awareness  of  preferred  personal  learning  styles  and  searching  patterns.  Realizing  that 
one  is  frustrated  and  confused  by  attempts  to  solve  certain  kinds  of  problems  is  another 
example  of  this  aspect  of  knowledge. 

Task  variables  refer  to  knowledge  about  4he  nature  of  the  tasks,  for  example,  knowing 
that  one  task  is  easier  or  more  difficult  than  another.  Knowledge  of  task  variables  implies 
that  the  individual  knows  the  type  of  cognitive  demands  required  in  certain  instructional 
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settings  and  how  the  nature  of  the  task  affects  the  manner  in  which  one  should  deal  with  it. 
For  example,  a  person  with  an  awareness  of  task  variables  may  know  that  it  is  generally 
more  difficult  to  synthesize  information  than  simply  to  remember  it. 

Strategy  variables  include  knowledge  of  how  to  use  a  strategy,  or  what  strategies  are 
available,  or  how  weli  a  btrategy  works.  When  an  individual  knows  what  strategies  are  most 
effective  to  obtain  the  best  learning  results,  then  he/she  is  said  to  possess  knowledge  about 
strategy  variables.  For  example,  in  most  learning  environments,  including  hypermedia- 
based  ones,  strategically  aware  individuals  might  plan  ahead  of  time,  monitor  their 
processes,  ask  themselves  why  certain  steps  °re  taken,  and  evaluate  their  performance. 

Flavell  (1987)  suggested  that  these  three  metacognitive  knowledge  variables  interact  in 
important  ways.  For  example,  individuals  might  realize  or  believe  that  certain  types  of  tasks  are 
very  easy  or  difficult  for  them  and  create  appropriate  strategies  that  work  especially  well  for 
them  and  the  task  in  a  certain  environment. 

Regulation  of  Cognition 

Regulation  of  cognition  is  a  metacognitive  process  defined  by  Flavell  (1979)  as  conscious 
monitoring  that  controls  and  regulates  both  cognitive  and  affective  processes.  Brown  (1987) 
argues  that  the  concept  of  metacognition  consists  of  activities  used  to  regulate  and  oversee 
learning.  According  to  Brown,  metacognitive  experiences  include  planning,  monitoring, 
evaluating  and  revising. 

Planning  involves  determining  an  overall  approach  to  completing  a  task  or  solving  a 
problem,  e.g.,  selecting  appropriate  strategies,  determining  what  sequence  should  be  taken, 
and  deciding  how  to  evaluate  progress  while  reaching  the  goals.  Monitoring  of  one's  own 
cognitive  processes  is  the  ongoing  process  that  one  goes  through  to  track  what  one  is  doing 
and  to  assess  success  while  implementing  plans.  Evaluating  one's  own  learning  processes 
involves  checking  to  see  what  has  been  done  and  deciding  if  the  same  approach  would  be 
good  for  similar  tasks  in  the  future.  This  process  of  evaluation  can  help  learners  build  a 
repertoire  of  strategies  that  they  can  call  upon  in  the  future  to  address  similar  situations, 
and  even  to  generate  new  approaches  in  novel  situations.  Revising  one's  own  learning 
procedures  involves  modifying  plans,  strategies,  learning  approaches,  and  goals.  In 
hypermedia-based  learning  environments,  users  must  be  able  to  plan  ahead  based  on 
knowledge  about  their  own  learning  capacities,  the  task,  and  strategy.  They  should  be  able 
to  monitor  their  own  comprehension  of  information,  evaluate  the  processes  used  and  the 
strategies  selected  to  deal  with  the  task,  and  modify  any  ineffective  strategies  and 
approaches. 

Recently,  the  affective  and  motivational  aspects  of  learning  have  received  increased 
attention  in  the  construct  of  metacognition.  Paris  and  Winograd's  (1990)  definition  of 
metacognition  includes  metacognitive  judgments,  beliefs,  and  choices.  Metacognitive 
judgments  refer  to  individuals'  ongoing  assessments  of  the  cognitive  characteristics  of  the 
learning  situation.  Metacognitive  beliefs  involve  expectations  that  reflect  affective  biases, 
self-concepts,  and. motivational  disposition.  These  two  components  are  important  because 
they  determine  which  tasks  learners  find  worthwhile  and  how  they  choose  to  engage  them 
(Paris  &  Winograd,  199U)»  Both  Schoenfeld's  (1985, 1987)  and  Lockhead's  (1985)  definitions 
of  metacognition  also  include  awareness  of  one's  beliefs  and  feelings  about  the  nature  of  the 
content  domain  and  the  self  as  a  learner. 

In  summary,  metacognitive  processes  imply  active  and  reflective  learners.  If  cognitive 
abilities  are  regarded  as  "enabling  variables,"  then  metacognitive  abilities  are  "organizing 
and  controlling  variables"  in  the  sense  that  they  contribute  to  the  selection,  sequencing, 
combination  and  regulation  of  cognitive  processes  (Rowe,  1988,  P,  228).  As  controlling 
variables,  they  enable  learners  to  organize  and  regulate  their  own  cognitive  learning 
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processes.  They  also  help  learners  select  and  regulate  the  direction,  duration,  intensity, 
range,  path,  and  speed  of  learning  functions  (Rowe,  1988;  Bruer,  1993).  As  a  result,  learners 
can  more  thoroughly  integrate  new  information  into  their  existing  knowledge  schemata, 
since  they  are  capable  of  selecting  exactly  what  is  needed  for  their  own  learning  systems. 
Therefore,  metacognitive  abilities  enhance  learners'  internal  control  of  cognitive  processes 
and  enable  learners  to  function  successfully  even  in  situations  in  which  they  must  make 
decisions  and  behavioral  adjustments. 

Metacognitive  Research 
Research  in  metacognition  can  be  organized  into  two  broad  categories.  The  first  involves 
studies  which  specify  how  metacognition  is  developed  naturally  throughout  life;  the  second 
concerns  the  research  into  the  trainability  of  metacognition.  For  the  purpose  of  stimulating 
future  research  on  integrating  metacognitive  training  within  hypermedia  systems,  the 
second  category  of  metacognitive  research  is  the  focus  of  the  discussion  here. 

Strategy  Training  Research 

The  research  in  this  field  assumes  that  the  more  people  know  about  strategies,  the 
greater  the  probability  they  will  use  the  appropriate  strategies  in  the  appropriate  situations 
(Vernon  &  Marie,  1984).  Most  of  the  early  research  has  attempted  to  teach  specific 
metamemory  knowledge  and  strategies.  For  example,  Brown  (1978)  summarized  a  serias  of 
studies  in  which  mildly  retarded  children  were  taught  to  know  when  to  select  missed  items 
when  learning  a  list  of  words,  and  when  they  were  ready  to  recall.  She  observed  from  these 
studies  that  the  training  effects  were  less  positive  than  expected  when  students  were  asked 
to  generalize  what  they  had  been  taught  to  different  situations.  She  thus  recommended  that 
more  general  metacognitive  strategies  should  be  taught. 

After  reviewing  relevant  literature  on  strategy  training,  Pressley,  Borkowski,  & 
O'Sullivan  (1984)  defined  three  methods  by  which  individuals  may  acquire  metacognitive 
knowledge  through  strategy  training.  The  firHt  iH  the  laissez-faire  method  in  which  no 
training  intervention  is  needed.  Learners  draw  conclusions  about  the  strategies  from  their 
applications  of  strategies.  PresN^y,  Levin  and  Ghatala  (cited  in  Duell,  1986)  did  a  series  of 
five  experiments  on  the  laissez-faire  method  with  itdultH  find  fifth-sixth  graders.  The 
findings  indicated  that  when  performing  new  task»  after  they  have  been  tested,  adult 
learners  selected  the  strategy  by  which  they  learned  the  most,  even  when  it  was  contrary  to 
what  they  had  been  told.  Fifth-  and  sixth-graders  tended  to  persist  in  using  the  strategy 
they  were  told  was  better  despite  the  contrary  evidence.  Only  when  they  were  provided 
information  on  how  the  strategies  should  he  used  correctly  did  they  select  the  more  effective 
strategies.  It  is  suggested  that  a  laissez-faire  approach  is  more  appropriate  for  adults  than 
for  children  (Pressley,  Borkowski,  &  0,Sullivan,  19H4), 

The  second  method  is  explicit  provision  of  metamemory  strategies.  This  method  assumes 
that  metacognitive  knowledge  can  be  explicitly  taught  through  providing  information 
concerning  the  utility  of  a  strategy,  the  importance  of  the  strategy,  and  information  on  how 
to  revise  the  strategy  to  meet  changing  task  demands.  Learners  are  most  likely  to  use 
strategies  spontaneously  when  they  are  given  knowledge  about  what,  how,  when,  where,  and 
why  certain  strategies  should  be  used  (Brown,  Campione,  &  Day,  1981).  This  information 
will  provide  learners  with  declarative  knowledge  (what),  procedural  knowledge  (how),  and 
conditional  knowledge  (when,  where  and  why)  of  strategies.  Pressley,  Borkowski  & 
O'Sullivan  (1984)  examined  the  effects  of  this  method  on  transfer.  They  provided  fifth  and 
sixth  graders  with  a  memory  task  (word  pairing  and  Latin  vocabulary  definitions).  The 
experimental  group  was  provided  instruction  telling  them  both  when  the  strategy  could  be 
used  appropriately  and  how  it  could  be  applied  to  other  situations  as  well  as  applying  the 
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strategy  to  three  diverse  situations.  Control  group  students  either  were  told  when  and  how 
to  apply  the  strategies  or  were  asked  to  apply  the  strategies,  but  were  not  given  the  combined 
treatment.  The  experimental  group  performed  better  on  a  transfer  test  consisting  of  new 
vocabulary  words. 

The  third  method  is  metamemory  acquisition  procedures.  It  requires  learners  to 
understand  how  the  strategy  works  by  experiencing  instructions  designed  to  help  learners 
derive  information  about  the  strategy  being  taught  (Duell,  1986).  This  method  assumes  that 
learners  must  be  taught  to  acquire  their  own  metacognitive  knowledge.  For  example,  Lodico, 
Ghatala,  Levin,  Pressley,  and  Bell  (1983)  taught  second  graders  to  monitor  or  evaluate  their 
performances  when  using  different  ways  of  accomplishing  the  same  task.  The  initial  training 
included  suggesting  that  there  were  many  ways  to  play  a  game  and  in  order  to  play  well, 
"they  must  select  the  method  that  allows  them  to  do  better"  (p.  267).  The  subjects  were  given 
two  tasks,  drawing  circles  and  remembering  a  list  of  letters.  Following  each  pair  of 
activities,  subjects  given  monitoring  instruction  were  asked  questions  which  made  them 
evaluate  their  relative  performances.  This  helped  them  identify  which  jf  the  two  activities 
was  more  effective  for  that  task  and  why.  Control  subjects  practiced  the  same  stravegies  as 
the  experimental  group,  but  received  no  feedback  and  practice  on  the  value  of  these 
strategies.  Students  in  the  experimental  group  recalled  much  better  than  those  in  the 
control  group.  They  also  outperformed  control  group  students  in  giving  the  reasons  for 
effectiveness  of  the  strategy  and  selecting  appropriate  strategy  for  learning.  However,  it  was 
also  suggested  by  Pressley,  Borkowski.  and  O'Sullivan  (1984)  that  this  method  may  not 
necessarily  bo  effective  for  all  training  because  the  memory  tasks  used  were  limited  to 
discrete  items. 

In  nummary ,  both  cnildren  and  adults  have  been  taught  a  variety  of  metacognitive 
strategies  successfully.  Research  has  demonstrated  the  effectiveness  of  informed  training,  in 
which  learner*  are  told  about  when  a  strategy  may  be  used,  its  application  in  a  variety  of 
situations,  its  effectiveness  for  producing  better  learning,  and  the  modification  of 
inappropriate  plans  and  strategics.  Metacognitive  strategy  training,  especially  metamemory 
acquisition  training  method,  has  been  shown  to  be  successful.  However,  all  of  these  strategy 
training  approaches  should  be  adopted  with  caution.  Sternberg  and  Wagner  (1982),  thus, 
propose  four  points  that  may  limit  the  effectivenessof  metacognitive  training  through 
strategy  training:  (1)  lurgescnlc  training  may  be  impractical  and  not  economic;  (2) 
externally  imposed  inetncognilivc  strategy  training  may  be  less  effective  than  the  strategies 
spontaneously  generated  by  students;  CD  students  may  not  elect  to  use  a  strategy  despite  its 
demonstrated  effectiveness;  and  (  I)  to  be  effective,  strategies  must  be  so  well  learned  that 
they  do  not  interfere  with  art  mi)  learning. 


The  majority  of  self  explanation  studies  are  derived  from  the  metacognitive  process 
perspectives  whirh  support  the  Importance  nf  active  and  self- generative  processing  of 
information  for  a  person  to  become  an  Independent  thinker.  Learners  can  develop  their  own 
controlling  and  monitoring  systems  It v  providing  self-explanations  for  what  they  are  going  to 
do,  their  reasons  for  doing  it  and  how  the  process  has  been  modified  (Campione,  Brown,  & 
Council,  IHHH).  It  lias  also  Keen  suggested  by  several  other  researchers  that  self-explanation 
training  can  induce  Independent  mctarngnitlve  processes,  i.e.  processes  directed  at 
understanding  of  one's  own  solution  process,  rather  than  relying  on  outside  cues  of  what  has 
been  done  and  how  it  has  been  dune,  which  leads  to  positive  transfer  of  skills  (Alhum- Heath 
&DiVcsta,  iilHfl;  lli'ttv.  I1IHM,  Herald)  t'olettu,  IIHHI).  This  position  implies  that  explaining 
one's  awn  processes  may  help  learncm  obtain  declarative  knowledge  (what),  procedural 
knowledge  (how),  and  conditional  knowledge  (when,  where,  why)  of  the  strategies  they  use 


during  the  process  of  self-discovery  learning,  which  in  turn  may  lead  to  the  development  of 
independent  metacognitive  abilities. 

There  are  several  studies  using  this  approach  to  metacognitive  training  which  report 
positive  transfer  of  skills.  Most  of  these  studies  are  from  the  literature  on  problem  solving. 
Berry  and  Broad  bent  (1987)  conducted  three  experiments  with  college  students  in 
invest igating  the  effects  of  two  different  types  of  explanation  training  in  a  computer  system 
on  the  performance  of  a  complex  search  task.  The  task  involved  determining  which  of  a  set 
of  factories  was  responsible  for  polluting  a  river  by  testing  the  river  for  the  presence  or 
absence  of  various  pollutants.  The  two  types  of  explanation  training  were  (1)  an  explanation 
provided  by  the  system  at  the  start  and  the  middle  of  each  trial;  and  (2)  a  condition  in  which 
subjects  were  allowed  to  auk  "why"  each  computer  recommendation  was  made.  Experiment  1 
showed  Mint  subjects  who  received  "why11  explanations  both  at  the  start  and  the  middle 
points  of  trials  performed  better  on  a  transfer  task  than  did  subjects  who  received  the 
explanation  only  at  the  Htart  of  each  trial.  Experiment  2  showed  that  subjects  who  were 
required  to  explain  "why"  following  the  explanations  provided  by  the  system  performed 
significantly  better  than  did  subjects  who  were  required  to  only  explain  "why",  but  received 
no  explanations  from  the  system,  Finally,  experiment  3  showed  that  subjects  who  received 
multiple  "why"  explanations  and  were  required  to  explain  "why"  at  the  time  when  the 
necessary  inferences  had  to  be  made  were  superior  on  two  subsequent  unaided  transfer 
tasks.  It  was  suggested  by  Berry  and  Broadbent  (1987)  that  the  amount,  level  of  specificity 
and  timing  of  explanation  is  important  for  the  success  of  transfer.  These  studies  also 
indicate  that  a  combination  of  external  and  internal  explanations  is  necessary. 

Another  group  of  studies  on  self-explanation  dealt  with  puzzle  types  of  problem  solving. 
Several  researchers  found  that  subjects  who  were  required  to  give  reasons  for  their  moves 
and  explain  how  decisions  were  made  outperformed  control  subjects  on  transfer  tasks  (Gagne 
&  Smith,  19B2;  Ahlum  Hcath  &  Divesta,  1986;  Beradi-Coletta,  1990).  These  subjects  made 
fewer  moves  and  got  more  correct  answers.  It  is  suggested  that  predictive  explanation  about 
possible  performance  in  hypothetical  situations  has  the  worst  effects  in  transfer  (Ericsson  & 
Simon,  1980).  Retrospective  explanation  on  what  one  did  after  the  event  is  little  better  than 
the  predictive  explanation.  But,  concurrent  explanation  is  the  best  in  helping  learning 
(Brown  &  Kane,  1988;  Chi,  Rasnok,  Lewis,  Itciimmii,  &  ft  laser,  1989). 

One  possible  reason  for  these  results  is  that  requiring  problem  solvers  to  explain  their 
thought  processes  while  solving  problems  "promotes  monitoring,  plnnfulness,  evaluation  and 
attention  to  problem  features,  and  the  sharpening  of  siii'h  executive  processes  yields  the 
more  efficient  performance"  (Dotnlnownkl,  IWM),  p. Ill  ft).  Another  possible  reason  is  that 
when  learners  have  reasons  lor  learning,  their  IntrlnMlc  wit  infliction  and  ownership  of 
knowledge  will  also  be  enhanced  (Hrown,  1H8M;  (loodimw,  11188),  People  will  become 
intentional  learners  when  they  linve  personal  Interest  in  learning.  Therefore*,  the  above 
approach  may  even  have  a  positive  effect  mi  motivation  ho  that  learners  believe  deep 
understanding  to  he  personally  valuable  and  at  e  willing  to  engage  In  the  self-explanation 
activities  .        However,  it  should  also  he  noted  that  the  pioecns  of  self -explanation  about 
what  one  is  doing  and  thinking  ran  he  difficult  for  many  eollegr  age  students,  especially  in 
dealing  with  complicated  tanks  (Hrown,  I !•  VM)  Additionally,  self  explanation  ran  have 
negative  effects  on  learning  when  the  requirement  for  self  explanation  competes  for  central 
processing  capacity  with  the  precedes  that  mind  be  reported  (Hrown,  IHH7),  The  following 
approached  proponed  by  <  Iruy  (I  Mil  I)  may  help  minimize  these  problems:  (I)  start  with  simple 
activities  so  that  learners  have  the  chance  to  practice  some  mctiicngiiltivc  skills  before  they 
attempt  more  difficult  problems;  [2)  use  two  levels  ot  sell  explanation,  first,  the  stops  of  the 
solution  process  for  specific  problems  should  he  recorded,  and  then  learners  should  reflect  on 
what  tliov  have  learned  from  this  process;  (U)  teachers  should  guide  at  the  beginning, 
providing  h  iirnei'H  with  specific  questions  to  direct  their  thinking  and  give  them  feedback 

.'.in 
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f 1 1 ii it 1 1  t  It i-l t  M'»i|nihH.  a  mill  t-ll  til iiilriitM  should  he  trained  to  provide  explanations  before 
c*ih  iHinti<i  ihM  1 1 m>  ipicst  Iumh  I  hut  link  them  hi  ntfiti*  reasons  lor  their  processes. 

liiUlttillvv  liiltivuilluii  Itmujih 

TIiIh  ii|i|tiiiurh  i*tii|iliiiH|/i'H  the  inipiiiluhnwi!  hotIiiI  interactions  and  treating  learners  as 
hi  Ihvi'mIImhIhih  In  Hip  development  of  iniMurtiKiiit  ivi1  skills  A  lot  of  research  in  this  field  has 
titki'ii  plinc  within  the  framework  of  Vy^n!  sky's  (IH7H)  theory  of  socially  mediated  learning. 
Thn  thi  ol  v  iihhiiiiiom  that  nil  imyrholoKlral  processes  are  initially  social*  shared  between 
people  (tcrn-hcrs,  piircnts,  peers),  ami  learning  in  transformed  through  interpersonal 
pun  i  hhi'h  (Hrnwii,  11JK7).  I'urin  and  Wiugogrnd  (HMK))  have  argued  that  metacognition  is  a 
viable  nihiliilnle  for  social  interactive  learning  because  insight  about  oneself  can  be  promoted 
hy  other  people  as  well  as  by  self-discovery. 

The  most  often  used  interactive  approach  to  teaching  metacognitive  skills  is  known  as 
reciprocal  training  that  has  students  interact  with  experts,  teachers,  and  parents.  The 
process  of  internalization  of  learning  is  gradual  from  external  support  to  internal  regulation. 
Teachers  usually  guide  the  problem  solving  activities;  gradually  students  and  teachers  share 
the  functions,  with  students  taking  the  initiative  and  the  teachers  correcting  when  it  is 
necessary;  finally  teachers  withdraw  the  guidance  to  the  students  and  act  primarily  as  a 
supportive  audience  (Brown,  1987).  Group  problem  solving  and  peer  interaction,  such  as 
discussion  and  controversy,  can  also  result  in  deep  understanding  of  what  one  is  learning 
(Brown,  1988).  It  is  believed  that  metacognitive  experiences  are  most  likely  to  occur  when 
students  are  required  to  explain  their  views,  defend  their  positions  and  evaluate  their 
processes  to  others,  especially  when  they  are  challenged  by  others  to  do  so  (Brown,  1988; 
Flavell,  1987;  Hatano  &  Inagaki,  1987).  It  is  when  students  are  forced  to  explain  their 
positions  and  reasons  to  others  that  they  con*°  to  understand  better  and  deeper  themselves 
(Hatano  &  Inagaki,  1987). 

Reviewing  the  literature  on  interactive  teaching  methods,  Reeve  and  Brown  (1985) 
described  two  programs  to  illustrate  this  approach.  One  program  was  conducted  by 
Palinscar  and  Brown  (1984)  on  improving  high  school  students1  reading  skills  and  the  other 
was  by  Scardamalia  and  Bereiter  (1982)  who  researched  improving  written  composition 
skills.  The  former  had  students  with  varying  levels  of  competence  and  an  adult  teacher  take 
turns  being  the  teacher  and  lend  a  dialogue  on  a  segment  of  text.  They  were  jointly  trying  to 
understand  and  remember  text  with  the  purpose  to  engage  students  in  four  important 
metacognitive  activities  relevant  to  effective  reading:  questioning,  clarifying,  summarizing 
and  predicting  events  in  the  text.  During  the  process,  the  teacher  and  students  gave 
feedback  to  each  other.  As  students  became  more  skilled,  the  teacher  decreased  her  level  of 
participation  and  acted  as  a  supportive  audience.  Over  a  several-week  training,  students 
"clearly  internalized  the  types  of  interactions  they  had  experienced,"  improving  not  only  the 
four  metacognitive  activities,  "hut  also  in  their  ability  to  assume  the  role  of  the  teacher, 
producing  their  own  questions  and  summaries,  and  evaluating  those  of  others."  (Reeve  & 
Brown,  1985;  p.  350).  Furthermore,  students1  feelings  of  personal  competence  and  control 
improved  dramatically.  They  also  concluded  that  teaching  metacognitive  skills  through 
direct  instruction  techniques  was  far  less  effective  than  reciprocal  teaching  (Palinscar  & 
Brown,  1984). 

The  second  program  had  novice  writers  think  aloud  while  writing  with  the  'lrpose  of 
externalizing  cognitive  processes  involved  in  writing  (Scardamalia  and  Bere*  •       °82).  Students 
were  treated  as  co -investigators  where  both  instructors  and  students  mode*.  -  »...nking  aloud 
procedures,  presented  cues  to  stimulate  self-questioning  during  the  planning  stage,  and  asked 
strategy  questions  on  how  to  resolve  controversial  ideals.  The  procedures  in  Scardamalia  and 
Bereiter's  study  resulted  "both  in  an  increased  ability  to  reflect  on  ideas  and  in  better- structured 
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compositions"  (Reeve  &  Brown,  1985;  P.  351). 

These  results  suggest  that  metacognitive  skills  can  be  enhanced  through  social 
interactive  processes  and  can  be  used  to  improve  academic  skills.  Such  social  interactive 
environments  can  be  "supported  by  computers  or  other  learners,  or  by  both"  (Brown,  1988;  p. 
319). 

In  summary,  improved  cognitive  functioning  has  been  shown  to  be  related  to  students' 
involvement  in  metacognitive  experiences  (Brown,  1987).  The  research  from  these  three 
metacognitive  training  approaches  has  also  illustrated  that  the  development  of  internal  self- 
regulation  is  central  to  metacognitive  development.  Moreover,  these  studies  indicate  that 
any  external  support,  regardless  of  form,  will  be  effective  for  metacognitive  development  only 
when  it  becomes  internalized  by  lcnrncrH.  Through  the  internalization  process,  individuals 
will  be  capable  of  generating  strategic  and  other  support  for  themselves.  Under  these 
enhanced  self-management  systems,  individuals  not  only  learn  how  to  accomplish  a 
particular  task,  but  also  learn  how  to  Icarn  (Brown,  Bransford,  Ferrara,  &  Campione,  1983). 
Therefore,  no  matter  what  approach  is  taken,  the  transition  from  external- regulated  learning 
to  internal  self- regulated  learning  should  ho  the  focus  for  metacognitive  training.  All  of  these 
training  approaches  should  have  important  implications  for  research  in  hypermedia-based 
instruction. 

Implications  to  Research  in  Hypermedia-Based  Learning  Environment 
What  emerges  from  the  review  of  the  literature  on  metacognitive  research  is  how 
learners  can  be  trained  metacognitively,  how  various  factors  may  influence  training,  how 
different  variables  may  work  jointly  to  influence  metacognitive  training,  and  the  cautions 
that  should  be  considered  while  employing  various  types  of  training.  The  fundamental 
importance  of  the  development  of  metacognitive  skills  is  that  learners  will  be  able  to  guide 
their  own  learning  process,  make  appropriate  decisions  independently  and  transfer  skills  to 
novel  situations.  As  a  metacognitive  learner,  one  "can  operate  as  a  self-correcting  system" 
(Rowe,  1988;  p.  228)  and  thus  is  capable  of  exploring  knowledge  independently  and 
generating  strategies  for  one's  own  need.  Moreover,  metacognition  may  also  enhance 
learners'  motivation  and  self-esteem.  Therefore,  metacognitive  knowledge  and  skills  are 
recognized  as  prerequisite  for  academic  and  real  life  success  (Rowe,  1988). 

The  most  often  mentioned  learning  problems  in  hypermedia  systems  are  learner 
cognitive  overload  and  disorientation  under  the  situations  where  learners  are  required  to 
control  their  own  learning  and  make  decisions  independently.   It  has  been  suggested  by 
several  researchers  that  students'  lack  of  metacognitive  knowledge  and  skills  may  be  one  of 
the  major  reasons  why  learning  is  hindered  in  situations  where  learners  are  given  control 
(Balajthy,  1990;  Steinherg,  1989).  When  lacking  metacognitive  knowledge  and  skills, 
individuals  cannot  identify  or  create  strategies  regulating  their  own  cognitive  processes,  and 
cannot  select,  adapt,  access,  combine  and  process  information  effectively  (Rowe,  1988).  If 
learners  are  capahle  of  effectively  controlling  and  regulating  their  own  learning  j  rocesses, 
they  should  not  he  confused  by  the  flexibility  and  wide  array  of  choices  of  the  hypermedia 
environment.  Therefore,  there  is  ii  need  to  identify  the  implications  of  the  research  in 
metacognitive  areas  to  the  learning  problems  defined  in  hypermedia  systems  and  suggest 
directions  for  future  research  that  merge  metacognition  into  the  design  and  development  of 
hypermedia -based  instruction. 

One  implication  arising  from  metacognitive  research  is  that  learner  control  should  be 
more  broadly  defined.  As  described  in  most  computer  literature,  learner  control  has  been 
limited  to  the  learners1  selection  and  sequencing  of  events  external  to  them.  However,  in 
addition  to  the  external  stimuli,  the  learners1  internal  systems  are  also  operating  and 
directing  learning  events.  It  is  the  promoting  of  internal  learner  control  that  needs  to  be 
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emphasized  when  talking  about  resolving  cognitive  overload  and  user  disorientation 
problems.  Findings  from  metacognitive  research  imply  that  effective  and  efficient 
performance  is  the  result  of  the  interaction  of  learners'  internal  systems  with  the  external 
stimuli  which  have  been  selected,  controlled  and  regulated  by  the  learners.  Those  learners 
who  have  developed  effective  internal  learning  systems  may  be  able  to  identify  what  they 
need  and  to  adjust  their  selection  and  control  of  display,  content,  or  any  external  stimuli 
based  on  their  internally  perceived  needs.  Therefore,  research  on  learner  control  issues  in 
hypermedia  systems  should  also  studv  learner  control  of  internal  cognitive  processing 
systems  as  well  as  internal  codification  schema. 

Investigations  of  learners'  internal  cognitive  processing  systems  should  examine  whether 
the  learners  use  the  most  effective  processing  strategies  available,  how  learners  manipulate 
their  own  cognitive  activities  to  adapt  into  new  instructional  settings,  and  what  types  of 
mental  activities  are  systematically  selected  by  certain  types  of  learners  while  interacting 
with  the  system.  Research  is  also  required  to  assess  the  attitudes,  motivation  and  attributes 
of  different  types  of  learners  as  they  use  hypermedia -based  instruction.  Another  question 
that  should  be  answered  is  how  the  system  can  be  structured  so  that  the  learners  will 
develop  more  effective  adaptive  models  of  their  own  to  enhance  performance  in  a  variety  of 
learning  situations. 

Controlling  one's  internal  cognitive  processes  is  an  important  aspect  of  metacognition. 
Therefore,  it  is  necessary  to  investigate  the  types  of  metacognitive  strategies  that  are 
important  for  different  types  of  learners  in  dealing  with  specific  topics  given  the  structures  of 
the  hypermedia  system.  What  metacognitive  skills  are  lacking  in  certain  types  of  learners 
while  dealing  with  specific  topics  under  specific  system  structures  and  why?  Among  those 
demonstrated  metacognitive  skills,  which  are  the  most  effective  for  the  types  of  learners, 
system  structures  and  topics  and  why?  Developmental  data  are  also  needed  to  categorize 
learners  with  different  metacognitive  abilities.  These  data  should  include  age,  maturity, 
cognitive  style,  learning  experiences  with  hypermedia  systems,  motivation  and  attribution  of 
learning  and  similar  variables. 

The  findings  that  simply  providing  learners  with  controls  over  manipulating  massive 
amount  of  unorganized  information,  and  letting  learners  explore  for  themselves,  will  not  be 
effective  in  improving  learning  also  has  implications  for  improving  metacognitive  abilities  in 
hypermedia-based  instruction.  The  real  challenge  for  hypermedia  designers  und  developers 
is  not  whether  to  impose  computer  control  or  not,  but  rather  how  to  take  advantage  of  the 
technology  to  help  learners  develop  their  own  consistent  internal  learning  capabilities  to 
maximize  those  resources  that  are  available. 

A  sophisticated  hypermedia-based  learning  system  should  provide  internal  adaptive 
functions  of  some  sort  to  provide  directions  for  such  internal  metacognitive  developments. 
Comprehension  monitoring  and  understanding  the  process  of  one's  own  or  other's  cognitive 
learning  should  be  encouraged.  Providing  strategy  advice  from  the  system  may  be  ineffective 
or  useless  unless  learners  are  informed  of  what  strategies  are  available  in  dealing  with  what 
t  ypes  of  problems,  how  to  monitor  their  own  use  of  the  strategies,  and  why,  when  and  where 
to  use  these  strategies.  However,  the  strategies  have  to  be  perceived  as  important  and  must 
be  prnctict  I  by  the  learners  in  order  to  be  effective.  Research  has  shown  that  learners  will 
not  automatically  become  involved  in  metacognitive  processes  unless  forced  to  do  so.  They 
may  he  aware  of  the  need,  hut  do  not  act  upon  it.  Thus,  the  systems  should  try  to  engage 
learners  in  exercising  metacognitive  strategies  as  naturally  as  possible.  Feedback  provided 
by  the  systems  should  reflect  the  nature  of  informed  training  in  which  learners  are 
encouraged  to  find  out  through  self-discovery  what,  how,  why,  where,  and  when  certain 
answers  come  into  being.  In  summary,  any  externally  provided  support  or  activities  should 
have  the  potential  to  facilitate  learners  to  transform  from  other-regulation  to  self-regulation 
of  the  learning  process. 
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Carefully  examining  research  findings  from  the  metacognitive  literature  may  stimulate 
ideas  for  designing  effective  and  efficient  hypermedia-based  instruction.  It  is  important  for 
hypermedia  researchers  to  identify  what  types  of  approaches  are  the  best  given  certain  types 
of  learners  in  dealing  with  specific  topics. 

Instructional  technologists  should  consider  how  to  creatively  merge  the  existing  effective 
metacognitive  training  procedures  into  the  design  and  development  of  effective  hypermedia- 
based  instruction  in  which  learners  make  optimnl  use  of  the  control  available  to  them.  If  a 
self-explanation  training  approach  is  taken,  it  is  crucial  to  determine  thu  amount,  the  level 
of  specificity,  and  the  timing  of  the  explanation**  Hint  lire  necessary  for  the  types  of  learners, 
topics,  and  systems.  It  is  also  important  to  investigate  what  types  uf  questions  human 
experts  ask,  what  types  of  explanation**  they  giwinle,  (Hid  when.  Hnch  investigations  may 
facilitate  design  of  appropriate  self-explanation  t  niiiiiiiK  f»r  liypi»l"iin»illn-biiH«d  instruction. 

If  a  strategy  training  approach  in  taken,  it  in  important  to  identify  the  typos  of  strategies 
that  should  be  taught  to  certain  types  of  leniiiei-H  in  dealing  with  rertiiin  topics,  and 
determine  the  order  in  which  the  strategy  training  should  he  delivered.  It  is  also  important 
to  find  out  what  types  of  strategy  training  (informed,  well'  discovery,  or  Hocial  interactive)  are 
most  effective  given  the  nature  uf  learner,  topie  anil  system  Htrueture.  The  major  concern  in 
employing  a  social  interactive  approach  is  accountability  of  urnup  members.  Educators 
continually  worry  that  one  member  may  participate  in  inotacoKiilt  ive  activities  while  the 
rest  will  ride  on  his  or  her  eoiiltnilH.  How  can  thin  approach  engage  each  member 
successfully  in  metacognitive  training?  The  import  mire  of  motivation  nnd  self-perceived 
competence  should  also  he  mlilrcHHed  in  any  type  of  metacognitive  training.  Under  what 
level  of  control,  what  types  uf  information  present  nf  ioiis  anil  system  structures,  will  learners 
generate  high  self- mot i vnl inn h  for  learning  and  interarting  with  the  environment?  What 
types  of  training  are  effective  in  improving  learners  motivation  and  self-confidence  in 
learning?  How  will  improved  motivation  affect  metacognitive  engagement?  Finally,  it  is 
important  to  determine  the  imparts  of  these  metacognitive  training  approaches  on  learners' 
applications  of  control  in  hypermedia  systems.  Will  metacognitive  training  improve  content 
specific  and  far  transfer  of  learning?  Will  metacognitive  training  enhance  learners'  retention 
of  information  and  understanding  of  the  relationships  among  concepts?  Will  such  training 
impact  learners'  consistency  of  strategy  application?  Will  it  improve  learners'  motivation  and 
self-confidence  in  learning  and  interacting  with  hypermedia  systems? 

Conclusions 

The  metacognitive  research  findings  are  promising  and  encouraging  in  support  of  a 
positive  relationship  between  learner  control  situations  and  learners'  improved  independent 
learning  abilities  as  well  as  their  motivation  to  learn.  Findings  from  metacognitive  research 
imply  that  learners'  internal  metacognitive  functioning  are  the  key  for  successful  learning 
under  learner  control  situations.  Hypermedia  researchers  should  investigate  how  to  use  the 
technology  to  help  learners  develop  their  own  consistent  metacognitive  systems  to  maximize 
the  use  of  the  learner  control  provided  by  the  system.  There  has  not  been  clear  guidance  as 
to  how  to  merge  metacognitive  theory  with  the  research  in  hypermedia-based  instruction  and 
learning.  It  is  not  clear  either  what  types  of  metacognitive  training  may  be  effective  for 
specific  types  of  learners  given  specific  topics  and  learning  environments.  Because  many 
fundamental  questions  related  to  metacognition  and  hypermedia-based  learning  still  remain 
to  he  answered,  the  field  is  a  fruitful  area  for  further  research  and  investigation. 
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The  study  of  collaboration  as  an  instructional  strategy  has  been  the  topic  of 
educational  research  for  more  than  seventy  years.  Laboratory  research  on  collaborative 
learning  dates  from  the  1920's  (Slavin,  1977),  intensified  in  the  1970*8  and  1980's,  and 
continues  to  be  an  active  topic  today  (Slavin,  1991). 

However,  the  successful  introduction  of  collaborative  methods  to  a  classroom 
requires  certain  changes  and  adjustments.  Students  must  be  taught  the  social  skills  needed 
to  work  in  groups;  teachers  must  teach  those  skills  and  guide  their  practice;  and  some 
adjustments  must  be  made  to  the  organization  of  the  class.  We  found  the  theoretical 
framework  of  diffusion  of  innovation  to  be  a  valuable  tool  in  explaining  how  collaborative 
activities  are  incorporated  into  the  classroom. 

Last  year,  we  began  a  preliminary  study  of  collaboration  in  the  higher  education 
classroom.  The  purpose  of  our  study  was  to  examine  how  collaboration  could  be  successfully 
incorporated  as  an  instructional  strategy  in  a  class  of  adult  learners.  Research  on  adult 
collaboration  in  the  classroom  usually  focuses  on  either  learning  outcomes  (for  example, 
Johnson  &  Johnson,  1987)  or  describes  specific  instructional  tactics  that  can  be  used  to 
promote  collaboration  (for  example,  Kagan,  1990).  Our  particular  interest  was  in  how 
collaboration  can  best  be  initially  introduced,  and  then-maintained,  as  an  instructional 
strategy. 


Studies  in  Collaborative  Learning 

Collaborative  learning  has  proven  to  be  a  powerful  instructional  strategy,  producing 
consistently  strong  effects  in  both  achievement  and  motivation  to  learn  (Slavin  et  al.f  1985; 
Johnson  &  Johnson,  1991;  Sharan,  1980;  Hamm  &  Adams,  1992).  Cooper  and  Mueck  (1988) 
define  collaboration  as    .  .  a  structured,  s/stematic  instructional  strategy  in  which  small 
groups  work  together  toward  a  common  goal."  One  point  to  note  from  this  definition  is  that 
there  is  a  common  goal  (to  solve  the  problem,  for  example),  but  there  may  or  may  not  be  a 
common  group  product.  Each  individual  may  produce  her  or  his  own  product  in  a 
collaborative  situation. 

Cooper  and  Mueck's  definition  is  compatible  with  Slavin's  (1991)  stipulation  that 
group  goals  and  individual  accountability  form  key  components  of  collaborative  learning 
situations.  Instructional  reward  systems  are  often  structured  to  provide  recognition  of  group 
achievement  or  to  assign  group  grades.  The  elements  of  group  dependency  and  individual 
accountability  also  form  key  components  in  the  work  of  Johnson  and  Johnson.  In  their 
exhaustive  research  (Johnson  &  Johnson,  1986)  on  collaboration,  they  describe  some  basic 
elements  of  collaborative  learning: 

Positive  Interdependence:  learners  feel  they  need  each  other  to  "sink  or 
swim"  in  completing  the  group's  task; 

Face-to- Face  Promotive  Interaction:  interaction  patterns  and  verbal 
exchanges  are  critical  to  the  learners1  success; 
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Individual  Accountability:  each  learner  must  be  helped  to  understand  and 
participate  in  the  task; 

Interpersonal  and  Small  Group  Skills:  learners  are  taught  and  practice 
abilities  such  as  trust,  communicati  ;n,  and  conflict  management; 

Group  Processing:  learners  need  the  time  and  structure  for  stepping  hack  to 
look  at  how  they  are  functioning  as  a  group.  This  metacognitive  process  can  be  aided 
by  observations  from  group  members  and  outside  observers. 


Our  Experience 

We  conducted  our  preliminary  study  in  a  gruduato-levul  class  nil  technology  for 
teachers.  The  class  was  composed  often  adult  learners  (t  line  men  atnl  aeveli  Women)  who 
were  all  current  or  preservice  teachers.  The  three-hour  cIuhh  met  onre  it  week  for  the  fifteen 
weeks  of  the  semester.  The  principle  content  of  the  cIbhh  whh  how  to  select,  ilcHigu  mill  uko 
instructional  technology  in  the  K-12  classroom  (video,  conitnutvlul  television,  email,  slides, 
interactive  learning  centers,  CBI,  etc.).  The  class  wiin  lecture  IhihcmI  and  often  Included  lnh 
sessions  on  the  development  and  use  of  the  technology.  Students  were  required  to  develop  a 
variety  of  media  projects  for  the  class. 

The  use  of  collaboration  was  considered  iih  ii  technological  competency.  The 
instructor  required  the  learners  to  experiment  with  collaboration  no  tluit  they  could  use  It 
themselves  in  their  professional  role  as  teachers,  ntul  mo  thot  they  could  tt»nt*li  thell  own 
students  to  collaborate.  Based  on  Johnson  and  JoIuihoIi'm  rcHcarcli  (IIIHfl),  we  ili*vlnoil  n 
three-part  program  to  train  the  class  in  collnhorution:  development  ol 1  IiiihIi*  collaboration 
skills,  coaching  and  practice  in  small  group  faco-to-fuce  structures,  and  motacognitivo  group 
processing  by  the  learners  of  their  collaborative  activities. 

Early  in  the  semester,  during  the  second  cIums  meeting,  we  conducted  n  three  hour 
training  session  to  instruct  the  group  in  some  of  the  hnnic  hkilln  of  collaboration  (conflict 
resolution,  listening  skills,  how  to  ask  effective  question h).  Throughout  flu1  Ni,mi,Hli,r,  tin* 
learners  were  asked  to  join  small  collaborative  groups  on  a  range  of  class  activities  I'or 
example,  groups  of  three  learners  completed  the  interactive  computer  program  Uregon  Trad 
together,  then  critiqued  the  program  as  a  team.  For  another  project,  students  were 
encouraged  to  work  in  groups  to  produce  a  short,  instructional  video.  They  received  n  group 
grade  for  this  project.  Two  people  chose  to  work  alone,  and  the  others  formed  groups  of  two 
or  three  people  to  produce  the  video. 

Over  the  couu.a  oi  the  semester,  the  class  was  required  to  work  in  collaborative 
groups  four  times.  On  each  occasion,  we  provided  individualized  coaching  and 
encouragement  on  their  collaborative  performance.  In  addition,  we  coached  the  groups  to 
discuHR  and  analyze  their  own  collaborative  behavior  at  the  end  of  each  HoSNion  (nee  I'tguri* 
1).  Group  processing  has  been  identified  by  Johnson,  Johnson,  Stanne,  and  Garibaldi  ( liMHI) 
as  an  important  contributor  to  the  success  and  productivity  of  groups. 
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COLLABORATION  JOURNAL 


Please  take  a  moment  to  reflect  on  how  your  group  used  the  skills  of  collaboration: 

•  Listening  •  Expressing  Empathy 

•  Questioning  *  Leading 

•  Resolving  Conflicts  •  Following 

•  Providing  Information  •  Energizing 

•  Clarifying  •  Encouraging 

•  Summarizing  *  Compromising 

•  Making  Predictions  *  Arguing 
9  Elaborating/Explaining 

1)  One  thing  I  think  our  group  did  well  in  this  session  was.  , . 

2)  One  thing  I  think  our  group  ought  to  work  on  is.  .  . 

3)  Collaboration  contributed  to  our  productivity  in  this  session  by.  .  . 


Figure  1,  This  is  the  format  the  groups  used  to  discuss  their  collaborative  activities.  Its 
purpose  was  to  promote  metacognitive  awareness  of  what  they  had  learned  and  practiced, 
and  to  articulate  their  own  goals  for  progress. 


One  of  our  most  consistent  observations  about  the  class  was  that  th(*  act  of 
collaboration  seemed  somewhat  foreign  to  the  students,  and,  off  and  on,  they  were  reluctant 
to  collaborate.  They  reported  that  they  know  about  collaboration,  and  that  they  had 
collaborated  successfully  in  other  contexts.  They  had  been  told  in  other  situations  that  it  was 
important  to  collaborate,  or  told  that  they  had  to,  but  had  never  formally  been  taught  how  to 
do  it  effectively.  Consequently,  they  lacked  organized  instruction,  examples,  practice  and 
feedback  in  the  skills  of  collaboration.  They  had  information,  but  lacked  the  experience. 

Their  lack  of  experience  should  not  be  surprising.  Our  society  has  not  placed  great 
cultural  value  or  invested  societal  resources  in  promoting  collaboration.  Judging  from  the 
literature  on  workplace  collaboration,  there  is  currently  a  desperate  need  to  teach  and 
encourage  adults  in  the  skills  of  collaboration.  For  example,  Lunsford  and  Ede  (1990) 
surveyed  professionals  in  various  disciplines  about  their  collaborative  writing  experiences. 
Of  the  700  respondents,  42  percent  answered  that  writing  as  part  of  a  team  or  group  was  "not 
too  productive"  or  "not  at  all  productive."  These  responses  are  especially  disconcerting  in  the 
face  of  numerous  surveys  of  professionals  confirming  that  collaborative  writing  is  a  major 
activity  in  the  workplace  (Penrose,  Bowman,  &  Flatley,1987;  Couture  &  Rymer,  1989; 
Killingsworth  and  Jones,  1989). 
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Collaborative  teamwork  nit  iUv  Job  Imih  bm»n  a  prime  topic  in  business  literature  over 
the  past  ten  yearn.  Miiliy  of  tin*  lu»Ht  k<  llin^  Ihihiiu>hb  books  of  the  decade  has  been  written  *n 
rcHpnnne  tn  tin?  apparent  f'ruHtt-at  iun  mill  iirgemy  of  introducing  teamwork  in  all  kinds  of 
nrgiim/atimiH  anil  fur  nil  levels  ot  employee**  (Kenno,  1990;  Peters,  1992;  Larson  &  LaFasto, 
I  WW;  Kimtei,  HIM:!;  Kiintoret  ill ,  1IHKI). 

Tin1  Denver  Puhlie  McIiooIh  have  faied  a  similar  difficulty  in  making  collaboration 
work.  Two  yeiii'ft  iiko,  t lit*  system  was  Mate  mandated  to  form  school-based  Collaborative 
Ihrittion  Making  TtnuiiH,  rumpimed  of  parents,  teachers,  community  members,  and  the  school 
principal.  According  to  tin*  latest  comprehensive  assessment,  one  of  the  most  difficult 
problem**  reported  by  tin*  teams  Iuih  been  their  lack  of  skills  and  experience  in  collaboration 
(( tarter  for  Quality  Hchooln,  1  BBiU.  The  trains  report  that  this  lack  of  skill  has  led  to 
inefficiency,  ineHectivoncHH,  and  high  turnover  in  team  membership. 


Collaboration  as  an  Innovation 

As  it  became  obvious  to  us  that  experience  in  collaboration  was  somewhat  new  to  the 
class  members,  we  shifted  our  thinking  to  look  at  collaboration  not  only  as  an  instructional 
strategy  but  as  an  innovation  as  well.  We  reviewed  the  literature  on  innovation  in  order  to 
understand  the  dynamics  of  this  process.  Everett  Rogers'  seminal  work  on  Diffusion  of 
Innovations  (1983)  provided  a  helpful  and  enlightening  structure  through  which  to  organize 
our  observations  of  the  classes1  alternating  ease  and  difficulty  in  implementing  collaboration. 

Rogers  defines  an  innovation  as  something  perceived  as  new  to  the  potential  adopters 
-  in  this  case,  our  learners    even  though  it  might  be  familiar  to  the  rest  of  the  world 
(Rogers,  1983).  According  to  the  diffusion  model,  an  innovation  often  involves  technology, 
both  hardware  (physical  tools)  and  software  (instructions  or  information).  In  both  these 
aspects,  the  diffusion  model  seemed  to  fit  the  situation  of  the  introduction  of  collaboration 
into  this  group  of  adult  learners.  The  model  proved  to  be  an  appropriate  choice  in  helping  us 
understand  the  experience  of  the  class  we  observed. 


The  Elements  of  Diffusion 

( Hit1  of  thi!  most  critical  concepts  of  the  diffusion  of  innovations  model  is  that  of  the 
elements  in  thi1  diffusion  of  an  innovation  (Rogers,  1983),  The  theory  states  that  the  more  of 
these  elements  that  are  present  in  any  particular  innovation,  the  more  likely  it  will  be 
adopted.  These  arc  tin*  questions  that  potential  adopters  consider  prior  to  deciding  whether 
to  adopt  thi1  innovation: 

Relative  Adv*Mltatlti-    Ih  the  innovation  seen  as  better  than  that  which  it  replaces?  More 
economical,  more  socially  prestigious,  more  convenient,  more  satisfying?  What  is  the  risk 
involved  in  adoption?  Ih  it  worth  the  change? 
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Observability.  An*  I  ho  results  of  the  innovation  visible  to  others,  so  that  they  can  see  how 
it  works  and  observe  the  consequences? 


Compatibility.  How'consistent  is  the  innovation  with  the  values,  past  experiences,  and 
needs  of  the  potential  adopters? 

Complexity.    Is  the  innovation  easy  to  understand,  use,  and  maintain?  Can  it  be 
explained  to  others? 

Trialflbilitv,    Can  the  innovation  be  tried  out  on  a  limited  basis? 


As  we  reviewed  these  five  elements,  and  applied  them  to  our  observations,  what  we 
were  seeing  began  to  make  more  sense.  Overall,  the  class  members  reported  being 
sometimes  ready  and  willing  to  collaborate  and  sometimes  reluctant;  willingness  to  adopt 
collaboration  also  varied  with  the  individuals.  Specific  issues  that  surfaced  in  the  class 
seemed  to  be  captured  by  Rogers1  elements  of  a  successful  adoption.  For  example: 

•  They  wondered  if  the  effort  was  worth  the  work  (Relative  Advantage). 

•  After  the  first  successful  collaboration,  they  were  more  willing  to  collaborate  the  second 
time  (Trialability), 

•  They  commented  that  there  were  too  fnnny  communication  rules  to  juggle  (Complexity). 

•  They  feared  they  wouldn't  have  time  to  collaborate  and  get  their  assignments  done 
(Relative  Advantage). 

•  They  said  that  collahnrutihu  was  hard  because  that  they  weren't  used  to  it  (Compatibility). 

•  Members  that  did  not  collaborate  as  much  commented  at  the  end  of  the  class  that  those 
who  did  collaborate  seemed  to  have  more  fun  doing  the  projects  (Observability). 

According  to  Rogers  theory,  concerns  and  feelings  like  these  will  typically  be 
experienced  by  the  potential  adopters  of  an  innovation  whether  we  and  they  are  aware  of 
them  or  not,  and  whether  we  and  they  want  to  deal  with  them  or  not.  We  believe  that 
considering  Rogers1  five  elements  will  assist  us  to  more  successfully  implement  collaboration 
by  helping  us  decide  how  and  when  to  best  adopt  and  implement  an  instructional  strategy 
like  collaboration. 

An  Innovation  Checklist 

Rogers'  theory  states  that  the  more  of  the  five  elements  that  are  present  in  any 
particular  innovation,  the  more  likely  the  innovation  will  be  adopted  (Rogers,  1983).  Looking 
back  on  our  experience  with  the  class,  wc  realized  that  if  collaboration  were  indeed  an 


innovation,  it  would  be  helpful  to  give  that  innovation  some  serious  consideration  before 
adopting  it.  We  thought  it  would  be  helpful  to  develop  a  structured  way  to  think  in  advance 
about  how  to  plan  for  successful  collaboration. 

We  offer  a  checklist  of  questions  for  several  reasons.  First  it  can  be  a  preparation 
tool  to  trigger  the  thinking  of  teachers  prior  to  implementing  collaboration.  We  need  to 
assess  whether  we  want  to  incorporate  collaboration,  how  much  to  incorporate  it,  and  how  to 
design  the  class  to  support  it.  Secondly,  we  found  that  collaboration  was  an  innovation  to  our 
learners,  and  that  it  was  important  to  prepare  them  thoroughly  as  well.  We  offer  the 
checklist  as  a  way  to  structure  a  conversation  with  the  learners  so  that  concerns  about  the 
innovation  can  be  considered,  articulated,  and  taken  into  account  when  designing 
instruction.  Here  are  our  initial  suggestions: 


Relative  Advantage:  (Is  it  worth  it  to  us?) 

What  is  the  advantage  to  our  using  collaboration  in  this  class?  What  will  be  the  advantage  to 
the  learners?  What  will  be  the  advantage  to  the  teacher?  How  can  we  support  our  goals 
while  using  collaborative  techniques?  Is  there  a  way  that  this  experience  in  collaboration  will 
be  useful  to  us  outside  of  this  class?  How  much  additional  time  and  effort  will  it  require  to 
prepare  for  and  implement  collaborative  activities?  Given  everything  else  we  are  doing  in 
this  class,  is  it  worth  our  time  and  effort?  Are  collaborative  activities  better  than  the  ones 
they  will  replace?  How  can  we  make  collaboration  interesting  or  fun? 

Observability:  (Can  wo  see  it  in  action  first?) 

('an  wo  observe  how  collaboration  i.s  being  used  in  other  classrooms  like  ours,  either  in 
person  or  through  other  media?  Can  we  obtain  information  about  collaboration  and  how  it 
works?  In  our  own  class,  can  we  observe  our  more  experienced  members  collaborate? 


Compatibility:  (Does  it  fit  for  us?) 

What  are  our  experiences  with  collaboration,  both  in  and  out  of  the  classroom?  Is 
collaboration  compatible  with  our  personal  and  educational  values?  What  are  our  group  and 
individual  needs  in  this  class  and  how  can  collaboration  contribute? 


Complexity:  (How  complicated  is  it?) 

How  can  we  keep  our  experiment  in  collaboration  simple  to  begin  with?  What  is  an  easy  way 
to  describe  collaboration  to  ourselves  and  others?  What  kind  of  training  and  coaching  will  be 
necessary  in  order  for  our  group  to  be  successful  in  a  collaborative  activity;  how  can  we  get 
that  training? 


Trialabilitv;  (Can  we  try  it  first  before  we  bet  the  ranch?) 

Is  there  a  way  to  try  out  collaboration  on  a  limited  basis  at  first?  Instead  of  redesigning  the 


509 


whole  class,  ran  we  incorporate  a  few  collaborative  experiences?  How  could  we  debrief  about 
our  cxpcricni'i'ti  in  order  to  correct  our  mistakes  and  capitalize  on  our  successes? 


Conclusion 

Although  educational  researchers  may  agree  that  collaboration  is  beneficial  to 
learning,  and  we  may  agree  as  a  society  that  collaboration  is  a  worthwhile  behavior,  the 
skills  of  collaboration  are  not  understood  and  practiced  on  a  regular  basis.  If  it  is  true  that 
collaboration  is  still  an  innovation  for  us  as  adults  in  both  the  classroom  and  the  workplace, 
then  models  that  explain  the  process  of  innovation  can  help  us  understand  how  to  promote 
the  innovations  we  want.  We  have  found  Rogers'  diffusion  of  innovations  model  to  be 
especially  useful  in  understanding  how  to  better  promote  an  instructional  innovation  like 
that  of  collaboration.  We  offer  a  checklist  of  questions  based  on  the  diffusion  theory  to 
prompt  thought  and  discussion  among  students  and  teachers  on  how  to  promote  the 
instructional  innovations  that  they  want  to  try. 
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Then:  is  much  Hiipptirf  lor  the  opinion  that  piluriif iiina!  '"chnulogy— especially  computer 
technology— could  have  a  major  positive  Impart  on  improving  tm  educational  system  (See 
The  National  Tank  Force  on  Kdueationul  TorliiioloKy.  19HH;  Hhn-.rtinr,  1990;  Sheingold  & 
Hadley,  1990;  United  States  Office  of  Technology  AHHOHHm„nt  |()TA|,  1988).  Indeed,  the 
availability  of  computers  lor  teaching  Iiuk  Increased  rupidly  (OTA,  1988).  Yet,  despite  the 
increased  availability  and  support  for  computers  in  teaching,  relatively  few  teachers  have 
integrated  them  into  their  teaching.  A  recent  survey  of  teachers  who  were  exceptional  users 
of  computers  for  teaching  averaged  only  about  one  such  teacher  per  school  (Sheingold  & 
hadley,  1990).  This  paucity  of  teachers  existed  even  though  the  availability  of  computers 
(59)  in  the  schools  surveyed  was  more  than  double  the  average  number  of  computers  (26) 
reported  available  for  schools  in  the  United  States  (Hecke"    989).  Extraordinary  availability 
of  computers  was  not  matched  by  an  abundance  of  extrao.  inary  users  of  computers.  The 
result  of  this  imbalance  is  that  computers  are  underutilized. 

This  research  was  undertaken  to  gain  insight  into  what  teacher  variables  may  be 
related  to  their  computer  use.  First,  computer  use  was  classified  as  a  process  of  the 
adoption  of  innovation  (see  for  example  Hall,  1982;  Rogers  1962,  1983;  Rogers  & 
Shoemaker,  1971;  Rutherford  &  Hall,  1982)  ur  more  specifically  Instructional 
Transformation  (Rieber  &  Welliver,  1989;  Welliver,  1990).     Secondly,  teachers'  computer 
use  was  examined  from  an  internal  perspective — of  the  influence  of  personological  variables 
to  levels  of  computer  use.  In  a  comprehensive  study  of  the  implementation  of  innovation  in 
education  (Berman  &  McLaughlin,  1977)  this  class  of  variables  was  excluded.  Yet, 
information  about  the  internal  variables—  the  predispositions  and  the  decision-making 
processes  of  the  person — may  be  the  most  valuable  for  influencing  behavior  or  performance 
(See  also  Coovert  &  Goldstein,  1980;  Gallo,  1986;  JordeBloom  &  Ford,  1988).  This 
viewpoint  was  supported  by  data  gathered  in  Sheingold  and  Hadley's  (1990)  survey  from 
which  teachers  who  were  exceptional  users  of  computers  for  teaching  had  been  characterized 
as  being  highly  motivated.  We  focused  on  motivation  since  it  appeared  to  be  a  dominant 
internal  characteristic-  From  within  the  broad  construct  of  motivation,  Expectancy  Theory 
(Vroom,  1964)  guided  the  selection  of  variables. 


Sample  and  Bt?»uig 

Practicing  teacher:?  and  prcservice  teachers  vv«»re  studied.  There  were  170  elementary 
school  teachers  constituting  the  "practicing  gr  .up."  Toacbers  were  eligible  to  participate  in 
the  study  if  they  and  their  schools  met  criteria  which  controlled  for  the  influence  of 
environmental  factors  and  ensured  th;i  'ho  of  availability  of  computers  was  uniform  for  the 
groups  of  teachers.  First,  the  teachers  tutight  a  variety  of  subjects.  This  was  neces3ary  to 
control  for  the  influence  of  t  he  dominance  of  compute-  use  for  a  specific  subject.  For  this 
reason  elementary  school  teacher?  were  studied  since  they  typically  teach  a  variety  of 
subjects.  Second,  computers  had  to  be  available  to  teachers.  Availability  was  defined  as  a 
computer-to-pupil  ratio  of  1  :  44.  (In  fact,  all  teachers'  schools  had  a  ratio  of  better  than  1  : 
25.)  Third,  computers  had  to  have  been  available  at  the  schools  for  at  least  three  years. 
Foui  schools  from  an  eastern  state  participated. 

There  were  lf?7  undergraduate  students  constituting  the  "preset vice"  group.  Two 
different  teacher  locus  of  control  scales  were  used  with  this  group.  One  half  used  the  Rose 
and  Modway  scale  (1981);  the  other  section  used  the  Maes  and  Anderson  scale  (1985). 
There  were  86  and  81  participants  respectively.  The  preservice  teachers  differed  from  the 
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practicing  teachers  also  in  that  they  were  instructed  to  complete  the  questionnaires  based 
on  their  expectations  (Marcinkiewicz  &  Grnbowski,  1992). 

Method 

Computer  use,  the  dependent  variable,  was  classified  into  three  ordered  levels:  Nonuse, 
Utilization,  and  Integration  using  the  Levels  of  Use  scale  (Marcinkiewicz,  in  press; 
Marcinkiewicz  &  Welliver,  1993)  based  on  the  Model  of  Instructional  Transformation.  The 
independent  variables  were:  innovativeness,  teacher  locus  of  control,  perceived  relevance  (of 
computers  to  teaching)  and  self-competence  (in  the  use  of  computers).  These  \ast  three 
variables  fit  the  formula  of  expectancy  theory  which  was  structured  to  predict  behavior 
based  on  three  elements:  1)  valence — a  goal  one  values  or  desires;  2)  expecf.ancy-the 
expectation  that  one's  effort  is  capable  of  achieving  some  performance;  and  3) 
instrumentality-the  belief  that  an  achieved  performance  results  in  attaining  the  valued  goal 
(Vroom,  1964).  The  performance  studied  was  the  use  of  computers  for  teaching.  The  goal  of 
the  performance  was  "quality  instruction."  Valence  of  providing  quality  instruction  was 
assumed  to  be  true  and  positive  for  teachers. 

Expectancy  was  assessed  by  teacher  locus  of  control  and  self-competence.  Teacher  locus 
of  control  reflected  teachers*  belief  in  their  influence  over  students9  outcomes.  The  Teacher 
Locus  of  Control  Scale  by  Rose  and  Medway  (1981)  and  the  Teacher  Role  Survey  by  Maes 
and  Anderson  (1985)  were  used.  Self -competence  reflected  teachers'  feelings  of  capability  in 
achieving  competence  in  using  the  computer  in  teaching.  An  originally  developed  measure 
was  used.  Instrumentality  was  assessed  by  perceived  relevance — teachers'  perception  of 
computer  use  as  relevant  to  teaching.  An  originally  developed  measure  was  used. 

Innovativeness  defined  as  "willingness  to  change"  was  assessed  using  the 
Innovativeness  Scale  by  Hurt,  Joseph,  and  Cook  (1977).  Data  were  collected  for  three 
relevant  demographic  variables:  age,  gender,  and  years  of  computer  experience.  An  attribute 
which  the  demographics  share  contrasts  them  importantly  with  the  other  variables — they 
are  not  amenable  to  influence  by  any  sort  of  intervention,  remediation,  or  staff  development. 
Yet,  they  may  be  significant  in  influencing  teachers'  computer  use.  All  variables  were 
assessed  using  a  composite  questionnaire. 

Results 

Table  1  shows  the  distribution  of  the  groups  by  levels  of  computer  use.  M.iro  practicing 
teachers  were  at  the  Nonuse  level  than  were  preservice  teachers.  For  presefvice  teachers,  the 
percent  a  Re  at  the  Nonuse  level  appears  to  be  nominal. 

Within  the  practicing  group,  the  Nonuse  and  Integration  levels  were  similar  roughly 
half  each  of  the  group  total.  Within  the  Preservice  group,  the  greatest  number  of  respondents 
was  at  the  Utilization  level. 
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100 

Nolo:  n  excludes  observations  with  missing  variables. 

To  determine  which  level  of  computer  use  distinguished  the  two  groups  statistically,  a 
chi-square  tiwt  was  computed.  The  2x3  chi  square  was  significant  c2(2,  n  =  337)  =  69.06,  p 

<  .01)  and  verified  the  apparent  differences  between  the  expected  and  the  reported  computer 
uno  of  the  groups.  Nonuse  distinguished  these  two  groups  the  most;  the  value  for  this  level 
contributed  the  most  to  the  overall  c^,  The  differences  in  the  groups  at  the  Utilization  level 
contributed  somewhat  to  the  c^,  The  differences  between  the  groups  for  the  Integration  level 
contributed  nominally. 

Univariate  intercorrelations  of  all  variables  were  also  computed.  The  correlations 
between  perceived  relevance  and  self-competence  were  high  for  both  the  practicing  (r  =  .53;  p 

<  .001)  and  the  preservice  group  (r  =  .376;  p  <  .001).  The  same  variables  were  both  most 
highly  correlated  with  computer  use  for  both  groups:  for  practicing  teachers,  self-competence 
(r  =  .42;  p  <  .001)  and  perceived  relevance  (r  =  .33;  p  <  .001);  for  preservice  teachers, 
perceived  relevance  (r  =  .28;  p  <  .01)  and  self-competence  (r  =  .23;  p  <  .05).  These 
relationships  may  support  the  theoretical  complementary  nature  of  these  two  variables. 

Stepwise  logistic  regressions  were  computed  to  identify  which  of  the  variables 
contributed  to  computer  use.  A  significance  level  of  p  =  .05  was  the  criterion  for  adding  and 
retaining  variables  in  the  regression  models.  For  the  practicing  teachers,  two  variables  were 
retained.  Self-competence  c2(2,  n  =  170)  =  19.07,  p  <  .001  and  innovativeness  c2(2,  n  =  170) 
=  5.12,  p  <  .05  were  identified  as  most  closely  related  to  teachers*  levels  of  computer  use. 
For  the  preservice  teachers,  perceived  relevance  of  computers  to  teaching  c^(\9  n  =  167)  = 
12.06,  p  <  .001  was  identified  as  most  closely  related  to  their  expected  levels  of  computer 
use.  No  other  variables  from  the  set  contributed  to  the  predictiveness  of  the  respective 
models. 

There  was  no  one  predictor  common  to  both  groups  of  teachers,  although,  the  high 
correlations  between  self-competence  and  perceived  relevance  for  both  groups  suggest  that 
these  variables  may  share  attributes. 

Analysis 

The  reported  levels  of  use  of  practicing  teachers  differs  widely  from  the  expected  levels  of 
use  of  preservice  teachers.  Most  importantly,  preservice  teachers  overwhelmingly  expect  to 
use  computers  for  teaching. 

As  for  the  correlates  and  predictors  of  computer  use,  it  appears  that  there  are  shared 
variables  for  both  groups.  This  suggests  consistency  of  the  variables  in  their  relationship  to 
teachers*  computers  use  whether  actual  or  expected. 
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Educational  importance 


This  research  was  motivated  by  the  concern  that  practicing  teachers  were  underutilizing 
computers,  and  that  perhaps  preservice  teachers'  expected  computer  use  would  mirror  the 
low  levels  of  the  practitioners.  The  results  show  that  preservice  teachers'  expectations  of 
computer  use  are  high.  This  may  be  an  indication  that  the  expectations  of  the  new 
generation  of  teachers  will  result  in  more  integrated  computer  use  in  teaching,  allowing  the 
promise  of  educational  computing  to  be  fulfilled. 

Computer  use  was  predicted  for  both  groups  by  variables  which  are  also  highly 
correlated,  namely,  self-competence  and  perceived  relevance.  In  planning  intervention, 
remediation,  or  staff  development — designing  instruction  for  computer  use,  these 
characteristics  might  deserve  addressing. 
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Perspectives 


There  is  much  support  for  the  opinion  that  educational  technology   especially  computer 
technology — could  have  a  major  positive  impact  on  improving  the  educational  HyHtem  (See 
The  National  Task  Force  on  Educational  Technology,  1986;  Shanker,  1990;  Sheingold  & 
Hadley,  1990;  United  States  Office  of  Technology  Assessment  [OTA],  1988).  Indeed,  the 
availability  of  computers  for  teaching  has  increased  rapidly  (OTA,  1988).  Yet,  despite  the 
increased  availability  and  support  for  computers  in  teaching,  relatively  few  teachers  have 
integrated  them  into  their  teaching.  A  recent  survey  of  teachers  who  were  exceptional  users 
of  computers  fcrf  teaching  averaged  only  about  one  such  teacher  per  school  (Sheingold  & 
Hadley,  1990).  This  paucity  of  teachers  existed  even  though  the  availability  of  computers 
(59)  in  the  schools  surveyed  was  more  than  double  the  average  number  of  computers  (26) 
reported  available  for  schools  in  the  United  States  (Becker,  1989).  Extraordinary  availability 
of  computers  was  not  matched  by  an  abundance  of  extraordinary  users  of  computers.  The 
result  of  this  imbalance  is  that  computers  are  underutilized. 

This  study  was  the  most  recent  in  a  series  in  which  the  relationship  of  teacher  variables 
to  their  adoption  of  computer  use  are  being  systematically  researched.  First,  computer  use 
was  classified  as  a  process  of  the  adoption  of  innovation  (see  for  example  Hall,  1982;  Rogers 
19(52,  1983;  Rogers  &  Shoemaker,  1971;  Rutherford  &  Hall,  1982)  or  more  specifically 
Instructional  Transformation  (Rieber  &  Welliver,  1989;  Welliver,  1990). 

Secondly,  teachers'  computer  use  was  examined  from  an  internal  perspective  -of  the 
influence  of  portumological  variables  to  levels  of  computer  use.  In  a  comprehensive  study  of 
the  implementation  of  innovation  in  education  (Berman  &  McLaughlin,  1977)  this  class  of 
variables  was  excluded.  Yet,  information  about  the  internal  variables — the  predispositions 
and  the  derision- making  processes  of  the  person — may  be  the  most  valuable  for  influencing 
behavior  or  performance  (See  also  Coovert  &  Goldstein,  1980;  Gallo,  1986;  Jordo-Hlootn  & 
Ford,  1HH8;  Mnrcinkicwic/,,  in  press;  Marcinkiewicz  &  Grabowski,  1992)).  This  viewpoint 
was  supported  by  data  gathered  in  Sheingold  and  Hadley's  (1990)  survey  from  which 
teachers  who  were  exceptional  users  of  computers  for  teaching  had  been  characterized  as 
being  highly  motivated.  From  within  the  broad  construct  of  motivation,  Expectancy  Theory 
(Vrooni,  \\)M)  guided  the  select  ion  of  variables.  In  this  version  of  the  research  series,  the 
variable  of  subjective  norms  was  included  (Ajzen  &  Fishbein,  1980)  because  it  suggests  a 
means  for  accounting  for  a  person's  decision  making  behavior  by  specifying  influential 
envrionmental  factors— significant  other  people, 


Sample  and  Setting 

There  were  1H8  elementary  school  teachers  from  twelve  schools  in  a  rural  midwestem 
state  who  participated  in  the  study.  Teachers  were  eligible  to  participate  if  they  and  their 
schools  met  criteria  which  controlled  for  the  influence  of  environmental  factors  and  ensured 
that  the  of  availability  of  computers  was  uniform  for  the  groups  of  teachers.  First,  the 
teachers  taught  a  variety  of  subjects.  This  was  necessary  to  control  for  the  influence  of  the 
dominance  of  computer  use  for  a  specific  subject.  For  this  reason  elementary  school  teachers 
were  studied  since  they  typically  teach  a  variety  of  subjects.  Second,  computers  had  to  be 
available  to  teachers.  Availability  was  defined  as  a  computer-to-pupil  ratio  of  1  :  12  (The 
state  average  ratio  was  1  :  9).  Third,  computers  had  to  have  been  available  at  the  schools 
for  at  least  three  years. 
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Method 


Computer  use,  the  dependent  variable,  was  classified  into  three  ordered  levels:  Nonuse, 
Utilization,  anil  Integration  using  the  Levels  of  Use  scale  (Marcinkiewicz  in  press; 
Mnreinkiewicy,  &  Welliver,  1993)  based  on  the  Model  of  Instructional  Transformation.  The 
independent  variubUw  were:  subjective  norms,  innovativeness,  perceived  relevance  (of 
compute™  to  teaching)  and  self-competence  (in  the  use  of  computers):  Subjective  norms  were 
measured  using  a  measure  based  on  the  steps  described  by  Fishbein  and  Ajzen  (1980). 
Four  significant  others — members  of  the  environment — were  identified  who  may  impact  on 
the  teachers  intent  to  use  computers  in  teaching.  Subjective  norms  reflects  a  person's  choice 
to  behave  based  on  the  influence  of  others. 

Innovativeness  defined  as  "willingness  to  change"  was  assessed  using  the 
Innovativeness  Scale  by  Hurt,  Joseph,  and  Cook  (1977).  Data  were  collected  for  three 
relevant  demographic  variables:  age,  gender,  and  years  of  computer  experience.  An  attribute 
which  the  demographics  share  contrasts  them  importantly  with  the  other  variables — they 
are  not  amenable  to  influence  by  any  sort  of  intervention,  remediation,  or  staff  development. 
Yet,  they  may  be  significant  in  influencing  teachers1  computer  use.  All  variables  were 
assessed  using  a  composite  questionnaire. 

The  last  two  variables,  perceived  relevance  and  self-tomptetence,  fit  the  formula  of 
expectancy  theory  which  was  structured  to  predict  behavior  based  on  three  elements: 
1)  valence — a  goal  one  values  or  desires;  2)  expectancy-the  expectation  that  one's  effort  is 
capable  of  achieving  some  performance;  and  3)  instrumentality-the  belief  that  an  achieved 
performance  results  in  attaining  the  valued  goal  (Vroom,  1964).  The  performance  studied 
wns  the  use  of  computers  for  teaching.  The  goal  of  the  performance  was  "quality  instruction," 
Valence  of  providing  quality  instruction  was  assumed  to  be  true  ami  positive  for  teachers, 

Teachers'  feelings  of  self. competence  of  capability  in  using  the  computer  in  teaching  were 
assessed  to  reflect  expectancy.  A  measure  developed  for  this  study  was  used, 
Instrumentality  was  assessed  by  perceived  relevance — teachers1  perception  of  computer  uhc 
<ih  relevant  to  teaching.  An  originally  developed  measure  was  used. 


Results 

Table  1  shows  the  distribution  of  categories  of  computer  use  for  the  sample, 
Table  1 

Distribution  of  Teachers  by  Levels  of  Computer  Use 


Level 

Number 

Percentage 

Nonuse 

31 

31 

Utilization 

66 

66 

Integration 

;i 

3 

Mulu.  The  observations  with  missing  values  were  not  included. 

A  univariate  intercnrrolntionnl  analysis  was  computed.  Subjective  norms, 
mnovativoticsH,  ami  perceived  relevance  were  equally  correlated  with  computer  one  (r  =  .26). 
The  relationships  may  suggest  a  theoretical  complementary  nature  of  these  variables  or 
colliuearity.  Other  high  correlations  that  emerged  were  between  perceived  relevance  and  self- 
competence  (r  ~  ,521)  perceived  relevance  and  innovativeness  (r  =  .42)  and  innovativeness 
and  self-competence  (r  =  ,211).  Age  and  computer  experience  were  moderately  correlate^ 
ir-M)). 
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A  partial  correlation  was  computed  to  remove  the  influence  of  the  overlapping  variables 
to  identify  the  variable  with  the  strongest  relationship  to  computer  use.  Subjective  norms 
was  shown  to  have  the  highest  partial  correlation  to  computer  use  (r  =.23).  The  correlations 
between  perceived  relevance  and  both  innovativeness  and  self-competence  remained  nearly 
unchanged  as  did  that  between  gender  and  innovativeness,  age,  and  experience. 

Intercorrelations  were  cast  to  test  for  relationships  among  variables.  Because  of  the 
categorical  nature  of  the  criterion  variable,  logistic  regression  procedures  were  computed  to 
identify  the  contribution  of  the  variables  to  teachers1  computer  use.  First,  the  predictiveness 
of  the  entire  set  of  variables  was  examined  c2(7,  n  =  100)  =  15.7,  p  <  .028.  Then,  a  stepwise 
procedure  was  computed  to  identify  the  predictiveness  of  individual  variables.  Subjective 
norms  was  retained,  c2(7,  n  =  100)  =  5.11,  p  <  .024.  No  other  variables  added  to  the 
predictiveness  of  the  logistic  regression  model.  Another  logistic  regression  was  computed  for 
the  simple  main  effects  of  subjective  norms, 
c2(lf  n  =  100)  =  7.09,p<.01. 

Analysis 

The  results  indicate  that  the  selected  internal  variables,  subjective  norms  is  predictive 
of  teachers'  levels  of  computer  use.  These  results  support  the  theories  which  suggested  the 
strength  of  its  being  an  indicator  of  behavior — the  theory  of  reasoned  action.  It  is  possible 
that  the  summary  of  studies  in  this  series  will  be  informative  for  the  characterization  of 
teachers  with  respect  to  computers. 


Educational  importance 

This  research  was  motivated  by  the  concern  that  teachers  were  underutilizing 
computers.  The  goal  was  to  identify  internal  variables  which  would  predict  computer  use.  In 
planning  intervention,  remediation,  or  staff  development — designing  instruction  for  computer 
use,  teachers'  subjective  norms  deserve  addressing.  It  is  also  important  to  consider  that  the 
construct  of  subjective  norms  is  based  on  one's  perceptions  of  the  expectations  of  others.  In 
light  of  this  assumption,  it  can  be  inferred  that  the  expectations  of  computer  use  from  among 
teachers*  significant  others' — principals,  colleauges,  students,  and  the  profession  -are 
influential  in  developing  teachers'  own  expectations  of  computer  use. 
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Classrooms  of  tin*  twi'nty-firwt  century  wi<l  experience  great  changes  due  to  emerging 
technologies  ami  the  development  of  new  instructional  delivery  systems.  Several  companies 
hiiVM  utti*inpt t*d  tu  incorporate  many  of  the  inHtructionai  strategies  of  effective  instruction 
into  computer-hascd  instructional  programs  called  integrated  learning  systems.  An 
integrated  learning  system  (IKS)  consists  of  computer  hardware  and  software  that  is 
generally  configured  as  ii  local  area  network  (see  Figure  1).  An  ILS  provides  a  comprehensive 
package  of  software  called  courseware  that  provides  computer-assisted  instruction  on  a 
network  of  computers  or  terminals.  The  courseware  also  includes  a  management  system 
that  triirkM  iniliviiluiil  learner  progress  and  adjusts  the  instruction  to  a  level  appropriate  for 
each  learner. 
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 Figure  1.  A 

Conceptual  Model  of  an  Integrated  Learning  System. 

Computer-based  instruction  is  having  a  profound  impact  on  both  the  field  of 
education  and  our  society.  ILSs  are  experiencing  enormous  popularity  and  sales  for  most  of 
the  major  systems  have  grown,  some  at  a  near  phenomenal  rate  (Sherry,  1992).  Large 
textbook  publishing  companies  have  purchased  ILS  companies  and  are  investing  heavily  in 
reshaping  them  for  the  future.  ILSs  account  for  a  large  portion  of  the  computer-based 
instructional  systems  currently  used  in  public  schools  and  many  educators  believe  that  these 
systems  will  continue  to  become  more  common  in  public  schools  (Sherry,  1990a), 

In  the  1950s  educational  researchers  attempted  to  solve  learning  problems  by 
applying  the  techniques  of  behavioral  analysis  as  theorized  by  B.F.  Skinner  through 
programmed  instruction.  The  concepts  of  programmed  instruction  were  then  applied  to 
crude  teaching  machines  which  first  appeared  in  the  late  1950s  and  the  early  1960s. 
Teaching  machines  and  programmed  instruction  were  used  throughout  the  decade  of  the 
1960s  by  colleges,  public  schools,  and  the  military  services.  Price  (1989)  submitted  that 
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although  research  on  programmed  instruction  *,en<:~all>  indicated  that  it  was  effective, 
programmed  instruction  never  achieved  a  Sigh  degree  of  popularity  because  it  was  tedious 
and  dull  and  did  not  fit  well  with  group  onsnted,  ffcyd-scbedule  school  settings. 

Computer-assisted  instruction  (CAI)  was  one  of  the  earliest  and  most  significant 
applications  of  computer  technology  to  education.  Tua  computer  industry  itself  was  among 
the  first  to  use  CAI  in  the  late  1950's  when  CAI  was  used  to  train  industry  personnel 
(Suppes  &  Macken,  1978),  At  a  time  when  programmed  instruction  was  the  focus  of 
educators  for  individualizing  instruction,  CAI  emerged  as  a  natural  integration  of  computer 
technology  and  the  programmed  instruction  movement  (Schoen  &  Hunt,  1977).  In  the  early 
ItttiO's  federal  funds  to  education  provided  a  stimulus  to  develop  CAI  models  (Atkinson  & 
Wilson,  19(>9). 

Early  efforts  to  use  computers  in  instruction  developed  from  the  guiding  principles  of 
programmed  instruction.  One  such  effort  was  a  project  at  Stanford  University  headed  by 
Patrick  Suppes  that  created  a  complete  system  for  computer-based  arithmetic  skills 
practice.  The  Stanford  Project  was  begun  in  1963  and  its  purpose  was  the  development  of  ° 
tutorial  system  to  provide  instruction  in  elementary  mathematics,  language  arts,  and 
reading.  By  the  end  of  the  second  year  of  operation  approximately  400  students  received 
daily  computer-assisted  instruction  in  either  reading  or  mathematics  (Suppes,  Jerman,  & 
Brian,  1968).  Computer  Curriculum  Corporation  was  formed  in  1967  as  a  direct 
consequence  of  the  Stanford  Project  and  the  need  for  curriculum-relevant  CAI  courseware. 
Computer  Curriculum  Corporation  marketed  computer-based  instruction  for  minicomputer 
systems.  Suppes  and  Morningstar  (1972)  validated  the  use  of  computers  as  effective 
teachers.  Suppes  and  Morningstar  maintained  that  the  creation  of  many  articulated 
programs  instead  of  isolated  topical  lessons  is  required  for  computers  to  be  used  effectively 
to  deliver  instruction. 

The  purpose  of  this  manuscript  is  to  examine  a  research  paradigm  that  is 
particularly  suited  to  experimentation  related  computer-based  instruction  and  integrated 
learning  systems.  This  model  for  research  is  relevant  when  an  experimental  treatment 
incorporates  an  individualized  or  adaptive  instructional  strategy.  Computer-based 
instruction  possesses  the  attributes  necessary  to  provide  both  the  adaptive  learning 
strategies  and  management  capabilities  for  highly  individualized  instruction.  The  model  is 
applied  to  an  instructional  treatment  using  a  popular  integrated  learning  system  by 
Computer  Curriculum  Corporation  (CCC). 


Theoretical  Bases  for  Research  on  Integrated  Lt*.  ning  Systems 

Recent  theory  development  in  human  learning  and  cognition  has  narrowed  the  focus 
of  educational  research  to  variables  and  conditions  that  are  directed  to  the  promotion  of 
learning  rather  than  the  improvement  of  teaching.  The  research  paradigm  for  educational 
technology  has  evolved  from  testing  an  instructional  technology,  media,  or  method  versus 
conventional  instruction  to  one  that  identifies  possible  instructional  variables  that  facilitate 
learning.  A  large  number  of  the  studies  concerning  the  use  of  computers  for  instruction  are 
described  as  media  comparison  studies  (Parry,  Thorkildsen,  Biery,  &  Macfarlane,  1985). 
These  studies  generally  emphasize  the  instructional  capabilities  of  the  media  with  little  or 
no  regard  to  the  components  of  instruction  and  learning  including  instructional  objectives, 
instructional  strategies,  learner  characteristics,  or  the  content  of  the  instruction. 

Salomon  and  Clark  (1977)  attribute  the  migration  of  educational  research  away  from 


media  comparison  studies  to  a  distinction  in  the  ranks  of  researchers  between  "research 
with"  and  "research  on"  media  (p.  102).  Salomon  and  Clark  further  explain  that  research 
with  media  considers  media  as  modes  of  stimulus  presentation  and  provides  little 
knowledge  about  the  specific  medium  used,  nor  does  it  provide  insights  about  how  learners 
learn.  Research  on  media  deals  with  the  relevant  attributes  of  media  that  interact  with 
individual  differences  to  promote  learning.      Clark  (1983)  makes  the  analogy  that  media 
are  vehicles  that  deliver  instruction  but  do  not  influence  student  achievement  any  more  than 
a  truck  that  delivers  groceries  causes  improvements  in  nutrition.  While  media  comparison 
studies  may  indicate  that  one  method  of  presentation  is  better  than  another  under  a  given 
set  of  circumstances  and  based  on  student  achievement  measures,  the  studies  fail  to 
demonstrate  which  attributes  of  the  instruction  are  responsible  for  the  results  (Stowitschek 
&  Stowitschek,  1984).  Media  comparison  studies  limit  the  extent  to  which  component 
variables  are  defined  and  operationalized  (Parry  et.  al.,  1985). 

Differences  in  learning  using  instructional  media  should  be  attributed  to  certain 
characteristics  of  the  instructional  medium  that  provide  conditions  to  facilitate  learning. 
Research  using  media  first  considers  a  medium's  attributes  and  determines  its  ability  to 
actually  relay  these  strategics.  What  is  implied  here  is  to  study  the  capabilities  of  the 
medium  and  determine  which  instructional  strategies  it  effectively  conveys.  If  computers  are 
utilized  as  a  modern  delivery  system  for  instruction,  research  questions  regarding  the 
effectiveness  of  computer-based  instruction  should  concern  variables  related  to  media 
attributes  and  instructional  design. 

One  of  the  primary  goals  of  computer-based  instruction  is  to  optimize  learning 
through  individualized  instruction  (McCombs,  Eschenbrenner,  &  O'Neil,  1973).  Computer- 
based  instruction  provides  adequate  technology  to  produce  a  learning  environment  that 
adapts  instructional  presentations  to  the  individual  differences  of  learners  than  would  be 
possible  under  instructor  management  alone. 

Since  computer-based  instruction  is  a  relatively  new  instructional  tool,  many 
researchers  and  educators  are  interested  in  its  potential  to  enhance  learning.  Much  of  the 
research  comparing  computer-based  instruction  to  other  instructional  methods  or  media 
provides  evidence  of  significant  learning  gains  using  computer-based  instruction.  Although 
the  research  indicates  that  computer-based  instruction  often  promotes  more  learning  than 
some  other  medium  or  method,  these  learning  gains  may  be  attributed  to  some  feature  of 
the  media  rather  than  the  media  itself.  Clark  (1983)  maintains  that  there  is  clear  evidence 
of  consistent  confounding  in  the  research  and  submits  that  the  confounding  variable  is  the 
design  of  the  instruction.  Computer-based  instruction  generally  requires  a  greater  effort  to 
design  the  presentation  than  the  comparative  media.  Clark  (1984)  concludes  that  learning 
gains  are  attributed  to  adequate  instructional  design  theory  and  practice  and  not  from  the 
medium  used  to  deliver  instruction. 

The  main  contributions  of  evaluative  and  comparative  research  related  to  computer- 
based  instruction  may  be  the  cost  benefits  and  learner  motivational  issues  to  be  considered 
when  using  computers  for  instructional  purposes.  The  most  constructive  insight  to  be 
derived  from  this  research  is  that  although  the  computer  as  a  medium  may  not  possess  any 
intrinsic  value  for  increasing  learner  achievement,  the  important  issues  for  research  with 
computer-based  instruction  are  concerned  with  the  design  variables  used  in  the  development 
of  computer  software. 


Definition  of  Term** 


MoHt  terminology  in  thin  mamiHcript  in  used  in  the  ronvuiitioiuil  hoiiho  ni' educational 
research.  However,  some  terniH  require  ndditionul  explunation  for  HpfJfif  usage  .n  tliiH 
manuscript. 

 Adaptive  Instruction.  An  adaptive  instructional  strategy  uses  one  or  more 

procedures  to  modify  instructional  activities  to  adjust  for  the  variance  in  the  aptitudes  of 
learners.  For  purposes  of  this  investigation  adaptive  instructional  strategies  are  inference- 
made  by  a  computer  program  about  the  aptitudes  of  learners  that  modifies  the  instructional 
presentation-usually  by  increases  or  decreases  in  the  grade  level,  quantity,  tutorial 
assistance,  or  speed  of  the  presentation.  Atkinson  (1976)  describes  adaptive  instruction  as 
a  process  in  which  the  sequence  of  instructional  presentation  and  activities  vary  as  a 
function  of  a  learner's  performance  history.  Hansen,  Ross,  and  Rakow  (1978)  define 
adaptive  instruction  as  a  corrective  instructional  process  that  facilitates  appropriate 
interaction  between  the  learner  and  the  learning  task  by  systematically  adapting  the 
allocation  of  learning  resources  to  the  learner's  aptitudes  and  recent  performance.  Hativa 
and  Lesgold  (1991)  state  that  instructional  software  systems  adapt  instruction  mainly  to 
learning-rate  differences.  Corno  and  Snow  (1986)  note  that  the  most  direct  manifestation  of 
cognitive  aptitude  differences  is  learning-rate  differences  and  that  instructional  designers 
usually  build  programs  that  adapt  to  learning-rate  differences  through  individualized  pacing 
with  repetition. 

 Aptitude.  Aptitude  is  generally  considered  to  be  a  broad,  multivariate  concept  that 

refers  to  the  individual  differences  among  learners  (Corno  &  Snow,  1986).  For  purposes  of 
this  study  aptitude  is  defined  as  a  variable  that  is  measured  in  terms  of  time.  According  to 
Carroll  (1963)  aptitude  is  the  amount  of  time  that  a  learner  is  willing  to  attend  to  a  learning 
task  in  relation  to  the  amount  of  time  required  by  the  learner  to  learn  the  task.  The 
amount  of  time  a  leaner  needs  to  learn  a  task  under  these  conditions  is  the  primary 
measure  of  the  variable  called  aptitude.  The  measurement  of  aptitude  is  inversely  related 
to  time  such  that  Carroll  noted  that  "the  shorter  the  time  needed  for  learning  the  higher  the 
aptitude"  (p.  726).  This  notion  of  aptitude  is  particulary  useful  for  research  on  integrated 
learning  systems  because  individual  differences  among  learners  may  be  quantified  as 
measures  of  engaged  learning  time  and  gains  as  recorded  by  the  management  program. 

Computer-based  instruction.  Many  discussions  of  computer-based  instruction 
distinguish  between  computer-assisted  instruction  (CAI)  and  computer-managed  instruction 
(CM I).  In  CAI  the  learner  receives  all  of  the  instruction  from  the  computer  including  tests 
and  performance  feedback  while  in  CMI  all  learner  testing  is  virtually  accomplished  on  the 
computer  but  the  learner  may  be  directed  to  other  media  for  further  study  (McCann,  1981). 
In  this  manuscript  computer-based  instruction  includes  both  CAI  and  CMI  or  any 
comprehensive  instructional  system  using  computer  hardware,  software,  and/or  computer 
networks  to  deliver  instruction. 

 Courseware.  Courseware  is  software  that  is  designed  specifically  for  educational 

and  training  purposes.  Courseware  is  a  program  or  bundle  of  programs  that  provides 
instruction  using  a  computer.  In  this  manuscript  courseware  includes  both  the  management 
and  uiHtructional  components  of  the  Hoftware. 


In  thin  manuscript  individualized  instruction  is  a  term 


used  to  denote  the  effects  of  adaptive  instructional  strategies.  Individualization  of 
instruction  takes  place  when  adaptive  instructional  strategies  account  for  individual 
differences  among  learners  and  adjust  instruction  to  a  level  and  mode  of  presentation  that  is 
appropriate  for  each  individual  learner. 

Strands.  Strand  is  a  term  used  by  the  CCC  integrated  learning  system  to  define  a 
set  of  exercises  in  one  content  area  and  arranged  in  order  of  increasing  difficulty.  A  strand 
contains  only  exercises  involving  a  particular  skill  or  content  area  and  spanning  several 
grade  levels.  Skills  are  linearly  ordered  within  each  strand  and  are  assigned  approximate 
grade-level  equivalents.  Skills  are  presented  in  a  cyclic  pattern  by  introducing  a  skill  at  a 
lower  grade  level  and  then  embedding  it  within  another  context  and  in  a  more  complex 
exercise  at  higher  grade  levels. 

Description  of  Integrated  Learning  Systems 

Integrated  learning  systems  are  also  called  integrated  instructional  systems  (IISs), 
integrated  teaching  systems  (ITSs),  or  intelligent  learning  systems  (ILSs).  Van  Horn  (1991) 
describes  ILSs  as  microcomputer  systems  that  contain  multi-year  curriculum  sequences. 
Maddux  and  Willis  (1992)  describe  ILSs  as  comprehensive  packages  of  software  and 
hardware  used  for  accomplishing  educational  goals,  usually  utilizing  a  local  area  network. 

An  ILS  compiles  an  instructional  program  at  the  level  a  learner  begins  instruction 
and  then  continuously  updates  the  instruction  according  to  a  cumulative  assessment  of 
learner  progress.   An  ILS  assesses  a  learner  initially  and  then  continuously  during 
instruction  and  iteratively  adapts  the  instruction  to  improve  acquisition  of  knowledge.  An 
ILS  generally  possesses  an  assortment  of  instructional  variables  and  conditions  to  adapt 
the  learning  interface  to  the  individual  differences  among  learners.  They  contend  that  an  ILS 
demonstrates  characteristics  generally  associated  with  an  experienced  teacher  such  as 
knowledge  and  application  of  learning  theory,  subject  matter  expertise,  appropriate 
assessment  and  measurement,  and  management  of  an  effective  and  efficient  learning 
environment. 

Norton  and  Resta  (1986)  suggest  that  ILSs  are  the  consolidation  of  two  branches  of 
instructionally  related  computing:  (1)  computer-assisted  instruction  (CAI)  utilizes  the 
computer  as  an  instructional  medium  to  provide  tutorials  and  drill  and  practice  of  related 
skill  areas;  and  (2)  computer-managed  instruction  (CMI)  utilizes  the  computer  as  a 
management  information  system  enabling  teachers  to  cope  with  the  record-keeping 
requirements  of  individualized  instruction. 

Sherry  (1990b)  provides  a  number  of  general  characteristics  which  describe  the 
design  and  function  of  ILSs: 

•  Computer-based  in  which  a  majority  of  instruction  is  accomplished  on  a  computer 
system. 

•A  networked  system  of  multiple  microcomputers  or  terminals. 
•A  management  system  which  collects  and  maintains  student  records,  prints 
reports,  and  provides  diagnostic/prescriptive  information  for  students  based  on 
individual  progress. 

•Courseware  that  spans  several  grade  levels  and  several  curricula  (math,  reading, 
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and  language) 

•  Ongoing  upgrades,  revisions,  and  expansion  of  existing  courseware. 

•  Links  its  lessons  to  an  accepted  standard  curriculum  and  suggests  sequences  for 
lessons. 

Maddux  and  Willis  (1992)  define  ILSs  as  comprehensive  packages  of  software  and 
hardware  used  for  accomplishing  educational  goals,  usually  utilizing  a  local  area  network. 
They  suggest  that  ILS  courseware  accomplishes  the  following  tasks: 

•  Assessment  and  diagnosis  of  student  skills 

•  Delivery  of  instruction 

•  Continuous  monitoring  of  student  performance  and  automatic  adjustment  of 
instruction 

•  Generation  of  student  and  class  performance  data  in  a  variety  of  formats 

Instruction  is  individualized  and  personalized  with  ILSs  and  teachers  are  able  to 
identify  and  individualize  remedial  activities.  According  to  Bailey  and  Lurnley  (1991)  the 
underlying  instructional  objectives  of  ILSs  are  based  on  individualized  instructional 
strategies.  These  strategies  include  random  generation  of  problems,  adjustment  of  the 
difficulty  and  sequence  of  problems  based  upon  learner  performance,  and  provision  of 
appropriate  and  immediate  feedback. 

Features  of  Best>Known  Integrated  Learning  Systems 

Several  companies  have  invested  heavily  in  ILSs  in  recent  years.  These  companies 
include  Computer  Curriculum  Corporation  now  owned  by  Paramount  Communications, 
Curriculum  Networking  Specialists,  Ideal  Learning,  Jostens  Learning  Corporation, 
MacMillan/McOraw  Hill,  New  Century  Education  Corporation,  PLATO  Education  Services- 
The  Rouch  Organization,  Wnsutch  Education,  and  WICAT  Systems.  While  each  ILS  may 
utilize  distinctive  iiiHtructional  strategies  and  management  approaches,  there  are  many 
hardware,  software,  and  iiiHtructional  features  and  characteristics  that  ILSs  have  in 
common.  Them?  t'onimonalitien  among  ILSs  are  examined  in  more  detail  in  this  section. 

Instruititmnl  ttlmmaia 

The  instructional  goals  of  the  various  ILSs  cover  a  wide  spectrum.  Some  systems 
are  designed  to  be  used  for  remediation,  others  for  comprehensive  instruction,  and  others  for 
development  of  higher-order  thinking  skills.  The  instructional  strategy  used  in  the 
development  of  lessons  can  be  categorized  as  either  "skills-based"  or  "concept-based" 
(Wilson,  1990).  Skills-based  programs  are  designed  to  provide  diagnostic  or  prescriptive 
intervention  for  remediation  of  precise  skills.  Concept -based  programs  are  designed  to 
develop  problem-solving  and  higher-order  thinking  skills. 

While  many  segments  of  ILSs  consist  of  drill  and  practice  and  tutorials,  a  number  of 
the  systems  offer  more  open-ended  activities  that  combine  basic  skills  development  with  an 
emphasis  on  problem  solving  and  higher-order  thinking  skills.  Many  systems  are 
incorporating  tools  for  research  and  exploration  by  adding  word  processors  and  reference 
tools  (electronic  encyclopedias,  atlases,  and  dictionaries)  to  their  systems. 

Maddux  and  Willis  (1992)  argue  that  ILSs  generally  follow  a  behavioral, 


competency-based  model  of  instruction  that  wan  more  popular  in  the  past  than  it  is  today. 
They  charge  that  ILSs  do  not  fit  well  with  the  cognitive  models  of  learning  and  teaching  that 
have  largely  replaced  behavioral  theories.  They  criticize  1I>4  vendors  for  commercial 
exploitation  and  being  unresponsive  to  changing  circumstances  and  developing  turnkey 
systems  that  reflect  the  desires  of  the  vendors  and  not  the  needs  of  teacherH,  parent**,  and 
students. 

White  (1992)  declares  that  one's  basic  educational  assumptions  determine  if  ILS 
instruction  is  "good"  education.  According  to  White,  ILS  instruction  providen  systematic 
exposure  to  the  curriculum,  individualized  pacing  and  review,  tracking  of  errors  and  re- 
exposure  to  more  instruction  in  order  to  reach  the  desired  mastery  level,  and  accurate  and 
comprehensive  records  of  each  learner's  progress,  ILSs  offer  systematic  drill  and  practice 
based  on  the  premise  that  some  learning  has  to  ho  practiced  until  it  is  automatic,  such  as 
reading,  vocabulary  acquisition,  and  estimating  of  quantities.  Tho  more  systematic  the 
practice,  the  sooner  the  skills  are  mastered. 

Shore  and  Johnson  (1992)  note  that  \IJR  vendors  follow  an  instructional  design  and 
development  process  that  attempts  to  ensure  that  the  instructional  strategies  of  the  ILS  are 
effective.  They  explain  that  the  product  development  process  at  Jostens  Learning 
Corporation  begins  by  gathering  information  from  teachers,  school  administrators,  and 
parents  in  dozens  of  school  systems  within  and  outside  the  United  States.  The  field  data  is 
evaluated  and  concepts  grounded  in  theoretical,  curricular,  and  instructional  design 
expertise  are  developed. 


Curriculum  Areas 

The  most  common  curriculum  among  ILSs  is  mathematics  followed  by  reading  or  a 
combination  of  reading  and  language  arts.  Science,  computer  skills,  computer  tool  skills 
(word  processing,  spreadsheet,  database),  social  studies  and  foreign  languages  are  included 
in  several  of  the  systems  (Wilson,  1990).  Several  systems  include  modules  for  GED 
preparation  and  ESL. 

Computer  Curriculum  Corporation  (CCC)  and  Jostens  learning  Corporation  software 
both  offer  learning  activities  appropriate  for  the  development  of  higher-order  thinking  skills 
and  problem  solving.  Computer  Networking  Specialists  (CNS)  specializes  in  reprogrammed 
versions  of  third-party  software  distributed  through  its  Classworks  management  system. 
Customers  select  software  from  a  list  that  includes  programs  from  many  leading  educational 
software  publishers.  New  Century  Education  Corporation  offers  a  special  reasoning  skills 
module  for  grades  three  through  nine.  PLATO  2000  software  offers  courseware  that 
includes  modules  which  teach  job  search  strategies,  life  coping  skills,  and  parenting  skills. 
Students  using  Wasatch  Education  System  software  have  access  to  an  online  spelling 
checker,  calculator,  graphing  function,  Logo  Language,  glossary,  database,  and  word 
processor  while  performing  learning  activities.  WICAT  Systems  courseware  is  designed  and 
keyed  to  complement  major  textbooks  and  several  national  and  statewide  tests  used  in 
schools  today, 

According  to  Power  On!  New  Tools  for  Teaching  and  Learning,  a  1988  report 
produced  by  the  U.S.  Congress's  Office  of  Technology  Assessment,  the  size  and  age  of  an  ILS 
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company  corresponds  to  the  grade  span  and  scope  of  the  courseware  offerings.  Bigger  and 
older  companies  offer  greater  coverage  while  newer  and  smaller  companies  target  fewer 
subject  areas  and  a  narrower  range  of  grade  levels. 

Generally,  lLSs  provide  sufficient  program  capabilities  to  manage  a  learner's 
program  and  measure  the  achievement  of  objectives.  However,  management  programs  vary 
greatly  among  the  ILSs.  The  management  approaches  range  from  simple  tracking  of  a 
student's  time  spent  on  a  given  learning  activity  to  a  complete  evaluation  of  a  student's 
programs  each  time  the  Enter  key  is  pressed  (Wilson,  1990). 

In  the  past  several  years  some  of  the  ILS  companies  made  significant  changes  in 
their  management  systems  in  response  to  customers1  concerns  for  more  flexibility  and 
openness.  Jastens  and  WICAT  developed  instructional  management  tools  to  integrate  a 
broad  set  of  curriculum  materials.  Several  companies  now  offer  systems  that  can 
simultaneously  manage  several  different  hardware  platforms,  Some  of  the  management 
systems  make  it  easy  for  teachers  to  create  customized,  multidisciplinary  course  sequences. 
Several  of  the  ILSs  allow  educators  the  ability  to  customize  student  reports. 

Some  companies  are  favoring  an  approach  that  leaves  more  management  control  in 
the  hands  of  teachers  over  an  approach  that  automatically  assesses  students  and  assigns 
lessons.  Additionally,  some  companies  have  added  new  testing  and  prescription  capabilities 
that  reflect  the  objectives  of  widely-used  standardized  tests  and  then  provide  individual 
prescriptions  specific  to  those  tests'  requirements  (Sherry,  1992). 


Learner  Characteristics 

Most  educators  subscribe  to  the  premise  that  individual  learning  differences, 
characteristics,  and  abilities  exist  among  learners  and  that  these  factors  should  be  taken 
into  account  in  instruction.  In  a  heterogenous  classroom  each  individual  has  a  different  set 
of  prior  skills  and  understandings.  Instructional  activities  are  best  constructed  on  an 
individual  basis.  ILSs  have  the  capability  to  evaluate  the  achievement  level  and  monitor 
the  progress  of  all  learners  using  the  system  across  all  grade  levels,  domains,  and  abilities. 
Learners  are  automatically  channeled  to  appropriate  lessons  by  the  ILS. 

Hardwire  Platforms 

The  ILS  industry  has  been  quick  to  respond  to  new  developments  in  the  personal 
computer  market.  The  majority  of  the  established  systems  operate  on  an  MS-DOS  network. 
However,  with  the  introduction  of  a  competitively  priced  color  Macintosh  several  ILSs  now 
offer  Mac  versions  of  their  systems.  A  few  companies  still  offer  Apple  II  courses,  but  most 
new  offerings  do  not  run  on  an  Apple  II.  Many  of  the  ILS  companies  are  moving  to  support 
their  software  under  a  Windows  environment  on  IBM  and  IBM-compatible  machines.  The 
trend  appears  to  be  that  within  a  few  years  all  ILS  vendors  will  offer  their  systems  on 
Windows  and  Macintosh  platforms  only  (Sherry,  1992). 


Multimedia.  Third  Party  Software,  and  Other  Features 


Multimedia  is  impacting  the  ILS  world  with  more  and  more  systems  offering  high 
quality  sound  effects,  digitized  human  speech  iind  music,  Kt  ill  photos,  it  ill  mat  It  m ,  nod  vhleo 
Several  ILSs  use  CD-ROM  to  deliver  encyclopedian  with  a  variety  nf  multimedia  elements 
and  some  systems  allow  students  to  record  and  playback  their  own  voices. 

Another  trend  in  the  ILS  industry  is  to  offer  networked  versions  til'  popular  stand 
alone  software  as  options  on  the  ILS.  Additionally,  many  ILH  companion  am  making  it 
easier  for  school  districts  to  launch  third-party  software  from  within  the  ILH.  Nome  systems 
are  going  a  step  further  by  allowing  third-party  software  to  ho  Included  anil  delivered  oh 
part  of  the  basic  course  activity  sequence.  This  npproaeh  allow*  teachers  to  Implement  a 
combination  of  ILS  courseware  and  third-party  software 

Jostens  and  WICAT  have  developed  instructional  management  Inula  that  Integrate 
a  wide  range  of  curriculum  materials.  These  tools  allow  educators  to  develop  a  curriculum 
design  that  links  ILS  courseware  with  school- district  objectives,  lesson*  plans,  ami  other  off- 
line materials.  Some  systems  allow  teachers  to  create  customized,  mult  idiscipli nary  course 
sequences  by  picking  and  choosing  from  available  ILS  offerings. 

ILS  Configurations 

Although  most  ILSs  are  designed  around  a  local  area  network,  several 
implementation  patterns  are  utilized  in  different  educational  settings.  In  a  lah 
configuration  all  the  computers  in  the  network  are  physically  located  in  one  room  or  area  of  a 
building.  A  coordinator  or  aide  may  schedule  students  or  classes  in  the  computer  lah  and 
manage  the  operations  of  the  computer  network  (hardware  and  software).  The  lah 
coordinator  or  aide  may  also  tutor  students  while  in  the  computer  lah. 

In  a  distributed  system  several  workstations  in  the  network  ore  located  in  various 
classrooms  around  the  school  building  or  school  system.  Since  computer  resources  are 
distributed  to  various  learning  areas,  a  distributed  system  requires  the  classroom  teacher  to 
manage  computer  resources. 

A  Research  Paradigm  for  Conducting  Research  with  ILSs 

As  the  costs  of  computing  technology  have  steadily  declined,  tho  availability  of 
computer-based  instruction  has  become  a  reality  for  classroom  applications.  Mont 
courseware  includes  advanced  features  such  as  graphics  and  animation*  interactivity,  and 
individualized  feedback.  Although  these  features  certainly  enhance  the  appeal  of  computer- 
based  instruction,  the  fundamental  questions  regarding  computer-based  instruction  ask  how 
well  the  instruction  actually  teaches  and  how  much  the  learners  actually  learn. 

Research  findings  that  examine  the  effectiveness  of  computer-based  instruction  arc1 
often  positive  (Bangert-Drowns,  Kulik,  &  Kulik,  1985;  Kulik,  Bangert,  and  Williams,  198U; 
Kulik  &  Kulik,  1987),  However,  these  findings  are  not  always  convincing  or  consistent 
enough  to  sanction  computer-based  instruction  as  better  than  conventional  instructional 
strategies  (Clark,  1983;  Salomon  &  Clark,  1977).  One  of  the  goals  of  research  related  to 
computer-based  instruction  is  to  determine  better  ways  to  utilize  its  attibutes,  features,  and 
delivery  capabilities  more  productively. 
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The  research  regarding  computer-based  instruction  is  generally  organized  around  the 
(Imil  role  iif  the  computer  in  providing  effective  instruction.  Some  researchers  advocate  the 
iihi'  uf  the  computer  as  a  problem-solving  aid  or  tutor  (CAI)  in  a  conventional  educational 
environment  while  other  researchers  champion  the  computer  in  the  role  of  managing 
instruction  (('Ml).  This  study  considers  the  effectiveness  of  of  both  the  CAI  and  CMI 
copahilitieH  of  computer-based  instruction  through  an  investigation  of  the  adaptive 
ihHt  motional  strategies  used  in  the  courseware  of  a  popular  ILS. 

The  main  assumption  of  this  research  parp-digm  is  that  one  of  the  most  powerful 
capabilities  of  computer-based  instruction  and,  sp  c:\fically  integrated  learning  systems,  is 
the  capacity  to  adapt  instruction  to  the  individual  differences  that  exist  among  learners. 
Through  individualized  or  adaptive  instruction  ILSs  have  the  potential  to  deliver  an 
appropriate  level  of  instruction  for  all  learners  and,  therefore,  improve  instructional  efficiency 
and  effective iiors.  The  research  model  described  in  this  manuscript  considers  the  adaptive 
learning  stratisgies  by  Computer  Curriculum  Corporation's  (CCC)  courseware.  The  CCC 
Curriculum  Profiles  (108*))  dnRerihe  CCC  courseware  as  "performance-based  instruction  that 
leads  to  rapid  ueudnmic  guins"  (p.  2)  uh  u  result  uf  an  individualization  process  that 
continuously  adapts  instruction  lor  each  learner. 

Adaptive  instructional  strategies  facilitate  learning  by  adjusting  learning  conditions 
to  the  individual  IciirniiiM  differences  muting  learners  (Tobias,  1976;  Rothen  &  Tennyson, 
H)7H).  A  computer  haned  adaptive  instructional  strategy  employs  on-line,  iterative 
algurif  Iiiuh  to  access  nn  extensive  data  base  and  adjust  the  learning  environment  to  the 
uniipic  li<n  1 1  hi  im  rhnrarteriMtics  and  individual  differences  of  each  learner  (Tennyson  & 
Itothnn,  1HY1M  The  uiuloi  Ivimm  assumption  of  a  computer-based  adaptive  instructional 
strategy  ih  that  the  effect  Ivonesn  of  the  learning  process  is  increased  over  conventional 
Mint  i  in  t  Kami  nii't  linilh  heciillHO  learners  receive  only  the  amount  of  instruction  required  to 
inaster  the  instructional  nh|iitives  (Uutheii  &  Tennyson,  1978). 

AccoiduiM  In  Itiitlicii  and  Tennyson  (1978)  the  primary  data  sources  for  implementing 
adil|itive  Iimii  illlifl  hI  ratlines  are  prctimk  measures  and  on-task  measures  of  learner 
achievement    Pi  chink  luentuircH  ace  used  to  diagnose  a  learner's  aptitude  for  learning 
|i.il th  ulisi  nlullii  anil  aie  derived  from  such  measures  as  scores  on  an  aptitude  or 
in  hlcvciuco!  t cut n  hi  pretcMt  scores  on  the  learning  task,  On-task  measures  are  based  un 
an  analv'tlh  m  iviihuilnai  id  Inni  nor  performance  during  the*  instruction.  Rothen  and 
Tonus  son  i  laini  Unit  |iinliiak  uml  on  limk  measures  of  learner  achievement  using  adaptive 
iiinti tu thin  ihlli  i  iiiiliidiinlliillv  liom  data  ohtnined  usiuir  conventional  instructional  methods, 
t  *i  his  mil  mihil  Hint  i  net  mi  nil  iin-l  IiihIm  identify  how  the  learner  answers  hut  do  not  identify  the 
Miiiuitive  nil  ill  1'iili'fi  Icudinit  III"  learner  to  tin*  answer,  Tennyson  and  Hothen  propose  that 
the  moid  tiupmliiut  roiiMlileialtou  of  nn  adaptive  lust  ruction  strategy  is  to  identify 
\v  \\  ho|u||ii nl  ciiiihom  tin  leal  iiiiim  mill  llnunliirc  decrease  the  probability  that  mistakes  will 
Ml  i  in 

The  adaptive  mIimIohioh  lieijuentlv  euiplnvnil  hy  connuercial  courseware  are  often 
»  enk  mill  IiiviiK  o  uiei  elv  ilntei uiliialiiiiiH  of  pnchiM  and  sei|iieiieiiiK  that  arn  derived 
Intel  nails  h\  I  In*  leiiinei  hi  eniei  nulls  hv  the  i  iiiii|iuler  |irograui  (Hiws  Kt  Morrison,  IUHH). 
I, mi  urn  iiiiit  I  ill  HliatctOt'M  aie  adaptive  mil  v  to  the  e*lcnt  that  learners  possess  the 
ii|ipinpi  hit  r  knowledge,  unit  in  it  \ .  ami  mot  ivalloii  lo  apply  nttccllve  |iiiImi<iiioiiIh  iihout  their 
It  hi m Mim  needs  (('mi  lor,  I liiviilunn,  Hi  Williaiast  HlHfi,  Hums,  lHH'l;  Hoks  &  Morrison,  1IINH, 
HleluhiUM,  WW)  Tin*  ehilioiale  iiimiiiKfuinut  sVHtcin  ol  II #Hm  pnr|iortedlv  pohmomm  Hie 
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capability  to  provide  a  comprehensive  strategy  for  adapting  instruction  based  on  both 
pretask  and  on-task  measures. 

Another  assumption  evident  in  this  research  model  is  that  individual  learners 
require  different  levels  of  instruction  based  on  their  aptitude  in  the  skills  being  taught  and 
that  measures  of  prior  learning  generally  constitute  strong  indicators  of  learning  needs  in 
inverse  relation  (e.g.  low  prior  achievement  indicates  a  need  for  high  instructional  support) 
(Tobias,  1976).  This  study  hypothesizes  that  an  ILS  adapts  the  magnitude  of  instructional 
support  for  each  learner  through  successive  revision  of  the  difficulty,  type,  quantity,  and 
sequence  of  instructional  activities  and  exercises  based  on  the  on-task  learning  trends  of  the 
learner. 

Stated  another  way,  the  problem  of  adaptive  instruction  is  to  diagnose  in  any  group 
of  learners  where  each  is  located  in  regard  to  individual  learning  characteristics  and  the 
nature  of  the  knowledge  and  performances  required  at  a  given  point  in  the  instructional 
process  (Seidel,  1971).  Once  this  assessment  is  made,  the  control  processes  of  the 
instructional  system  must  utilize  the  feedback  from  the  learner  to  continuously  refine  the 
estimate  of  the  learner's  progress.  The  focus  of  research  for  the  model  described  in  this 
manuscript  concerns  the  adequacy  of  the  adaptive  learning  strategies  utilized  by  a  model 
integrated  learning  system  to  accurately  assess  a  learner's  achievement  level  and  provide  an 
appropriate  amount  of  instructional  support. 

A  Micro  Theory  of  Adaptive  Instruction 

Mast  applied  individualized  instruction  models  or  programs  such  as  mastery 
learning  or  Keller's  (1968)  Personalized  System  of  Instruction  (PSI)  establish  conditions  for 
learning  based  on  high  instructional  support  that  features  the  availability  of  as  much  time 
and  resources  as  the  learner  requires  to  achieve  the  learning  objectives.  A  frequent  problem 
with  these  approaches  to  individualized  instruction  may  be  the  selection  of  too  much  support 
by  high  achievers  and  too  little  support  by  low  achievers  due  to  the  fact  that  personal 
prescriptions  as  to  how  much  instructional  support  is  necessary  for  each  learner  are  not 
provided,  leaving  it  to  the  learner  to  make  those  decisions  (Ross,  1984).  Interest  in  this 
problem  and  the  potential  that  computer-based  instructional  systems  possess  for  adapting 
instructional  support  to  individual  differences  among  learners  has  prompted  some 
researchers  to  propose  alternative  designs  of  individualized  instructional  models. 

Ross  and  Morrison  (1988)  developed  a  model  for  systematically  adapting  the  amount 
of  instructional  support  to  individuals.  The  pre-instructional  components  of  the  model 
consisted  of  selecting  predictor  variables,  developing  predictive  equations,  and  selecting 
appropriate  instructional  prescriptions.  The  instructional  components  consisted  of 
prearranging  and  administering  learning  materials  according  to  the  individual  learner's 
prescriptions,  administering  a  formative  lesson  posttest  upon  completion  of  each  lesson,  and 
use  of  the  lesson  posttest  to  refine  instruction.  The  Ross  and  Morrison  model  selected 
pretask  (entry)  variables  as  a  basis  for  predicting  learner  performance.  As  a  result  of  prior 
research  (Hansen,  Ross,  &  Rakow,  1978;  Ross,  1984;  Ross  &  Rakow,  1982)  Ross  and 
Morrison  developed  predictive  equations  about  learner  achievement  by  regressing  lesson 
subtest  scores  on  entry  variables. 

Data  analysis  in  educational  research  is  traditionally  performed  using  classical 
statistical  theories  that  compute  a  coefficient  based  on  some  sort  of  group  average.  When 
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researc  h  projects  examine  the  effects  of  individualized  instructional  strategies,  the  analysis 
must  compute  a  coefficient  based  on  factors  that  characterize  a  different  treatment  for  each 
individual  learner  because  the  treatment  is  different  for  each  individual  learner.  Macken, 
Suppes,  and  Zanotti  (1980)  explained  that  such  a  design  is  more  appropriate  for  research 
projects  that  analyze  individualized  instruction  because  this  design  considers  a  different 
treatment  for  each  individual  learner.  Macken  et  al.  argued  that  a  global  analysis  of  data 
involving  individualized  instruction  requires  operationalizing  on  factors  that  characterize  the 
individual  treatment  conditions. 

Suppes,  Fletcher,  and  Zanotti  (1975,  1976)  proposed  a  micro  theory  for  analyzing 
and  evaluating  individualized  instruction.  The  model  is  similar  to  the  Ross  and  Morrison 
(1988)  model.  In  the  Suppes  et  al.  model  the  amount  of  time  a  learner  spends  on  a  learning 
activity  is  a  function  of  the  learning  progress  made  by  the  learner.  A  learner's  achievement 
as  related  to  the  course  objectives  is  expressed  as  post-treatment  grade  placement.  This 
theory  was  tested  and  used  to  achieve  precise  individualization  of  instruction  both  in  the 
quantity  of  instruction  and  the  achievement  goals  for  each  learner  (Malone,  Suppes,  Macken, 
Zanotti,  &  Kanerva,  1979;  Suppes,  Macken,  &  Zanotti,  1978;  Suppes  et  al.,  1975,  1976). 

Suppes  et  al.  (1975,  1976)  micro  theory  of  individual  performance  expresses  a 
predictive  equation  about  expected  learner  outcomes  using  variables  based  on  the  individual 
performance  ui  each  learner.  Each  learner's  progress  through  a  curriculum,  the  learning 
trajectory,  is  expressed  by  a  family  of  curves  in  the  form 

y(t)  =  btk  +  a 

where  y(t)  is  a  learner's  grade  placement,  t  is  the  amount  of  time  spent  in  the  curriculum,  k 
is  a  parameter  estimated  for  a  particular  curriculum,  and  b  and  a  (slope  and  intercept)  are 
individually  estimated  for  each  learning  trajectory. 

Malone  et  al.  (1979)  refined  this  micro  theory  model  to  predict  a  future  trajectory 
based  on  observations  of  past  performance.  In  an  analysis  of  ten  possible  models  Malone  et 
al.  determined  that  a  model  in  which  the  learning  trajectory  increased  linearly  from  the  last 
observed  point  at  a  learning  rate  b  that  is  the  slope  of  a  line  determined  by  data  for  all 
learners  best  predicted  future  achievement.  The  learning  trajectory  for  this  model  is 
expressed  in  the  form 

y(t)  =  b(t-tr)  +  o(tr) 

where  y(t)  is  a  learner's  predicted  grade  placement  after  t  amount  of  time  spent  in  the 
curriculum,  h  is  the  slope  of  a  line  that  best  fits  all  the  points  for  all  students,  tr  is  the  time 
of  the  most  recent  observation,  and  a  is  the  grade  placement  at  the  intercept.  This  model 
basically  predicts  that  a  learner's  estimated  grade  placement  continues  to  rise  from  the  last 
observed  point  at  a  rate  that  is  about  the  average  rate  for  the  population  as  a  whole. 

A  crorullary  of  this  model  postulates  that  a  learner's  grade  placement  increases  with 
the  square  root  of  engaged  learning  time  was  also  determined  by  Malone  et  al.  (1979)  to 
effect  ively  predict  future  achievement.  This  model  takes  the  form 

y(t)  =  b(f5-tr^)  +  a(tr) 
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Malone  et  al.  (1979)  distinguished  two  uses  of  this  model  as  curve  fitting  and 
prediction.  They  explained  that  the  model  may  be  used  to  approximate  student  learning 
curves  based  on  data  from  an  entire  school  year  or  the  model  may  be  used  to  predict  grade 
placement  at  some  future  point  based  on  observed  data  for  part  of  the  school  year. 

A  critical  feature  of  the  adaptive  models  and  theories  advanced  by  Suppes  et  al. 
(1975,  1976),  Malone  et  al.  (1979),  Ross  and  Morrison  (1988),  and  Ross  and  Rakow  (1982) 
is  the  capability  to  formulate  prescriptions  systematically  tailored  to  the  inferred  needs  of 
individual  learners.  This  critical  feature  of  adaptive  instruction  provides  a  basis  for  the 
primary  research  objective  proposed  for  the  model  described  in  this  manuscript  that  it  is 
possible  to  determine  reliable  prescriptions  for  learner  achievement  when  an  instructional 
model  is  effectively  adapting  instruction.  Furthermore,  one  way  to  measure  the  effectiveness 
of  an  adaptive  learning  strategy  is  to  measure  the  reliability  of  predictions  about  learner 
achievement  based  on  observations  of  learners'  past  achievement  during  the  adaptive 
treatment.  Specifically,  this  manuscript  proposes  to  extend  the  micro  theory  research  of 
Suppes  et  al.  and  Malone  et  al.  by  correlating  the  predictive  quality  of  an  instructional 
treatment  with  the  adaptive  quality  of  an  instructional  treatment, 

The  present  research  proposal  identifies  and  describes  the  adaptive  processes  and 
strategies  programmed  in  the  CCC  courseware  and  hypothesizes  the  predictability  of  learner 
outcomes  based  on  the  adequacy  of  these  adaptive  mechanisms.  To  determine  the  adaptive 
quality  of  the  instruction  provided  by  the  CCC  integrated  learning  system,  several  research 
questions  are  considered.  Is  the  instructional  treatment  sufficiently  adaptive  to  formulate 
reliable  predictions  about  individual  learner  achievement?  Does  the  CCC  integrated 
learning  system  provide  a  better  estimate  of  learner  aptitude  than  an  external  measure  of 
achievement?  Does  increased  instructional  time  provide  a  more  reliable  prediction  of  learner 
achievement? 

CCC  Instructional  Design 

CCC  courseware  is  organized  into  strands  that  contain  a  sequence  of  exercises  from 
the  same  content  area.  Exercises  within  strands  are  grouped  into  equivalence  classes  and 
ordered  according  to  their  relative  difficulty,  (trade  levels  are  assigned  to  each  equivalence 
class  according  to  the  appearance  of  similar  exercises  in  elementary  textbooks  and  standard 
achievement  tests  (Malone,  et  al  ,  1979).  During  an  instructional  session  a  learner  receives  a 
random  mixture  of  exercises  from  all  the  strands  appropriate  for  that  learner's  grade  level 
and  then  the  difficulty  level  of  exercises  is  adjusted  to  the  learner's  achievement  level  in  the 
strand.  The  management  system  determines  a  weighted  average  of  the  learner's  grade 
placement  across  all  strands  in  a  course  (Macken  et  al.,  1980).  This  grade  placement  across 
all  the  strands  is  the  variable  that  is  of  particular  interest  for  this  study  in  determining  the 
adequacy  of  the  adaptive  strategies  employed  by  the  courseware. 

According  to  the  CCC  Curriculum  Profiles  (1989)  CCC  employs  a  mastery  learning 
model  in  their  integrated  learning  system.  The  CCC  learning  model  bases  proof  of  mastery 
on  answer  patterns  rather  than  on  a  percentage  of  correct  answers.  This  strategy  ensures 
that  each  learner  automatically  advances  within  a  skill  area  as  soon  as  mastery  is  achieved. 
Progression  through  a  skill  area  is  paced  by  the  learner's  actual  performance.  CCC 
maintains  that  this  performance-based  strategy  based  on  answer  patterns  keeps  learners 
fully  engaged  and  challenged.  The  Curriculum  Profiles  state  that  CCC  courseware  can 
produce  rapid  academic  gains  in  learners  as  a  result  of  an  individualization  process  that 
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produces  instruction  that  is  continuously  adapted  tor  each  learner.  Several  features  in  the 
design  of'CCC  courseware  contribute  ta  the  individualization  of  instruction: 

•The  foundation  of  the  instructional  strategies  employed  by  CCC  courseware  is  the 
availability  of  a  comprehensive  curricula  in  several  skill  areas.  In  order  to 
individualize  and  adapt  instruction  for  a  large  number  of  learners  working  at  a 
variety  of  skill  levels  a  vast  amount  of  instructional  material  is  required. 
•  Initial  placement  motion  adapts  the  beginning  level  of  instruction  to  learner 
performance.  During  the  first  ten  sessions  a  learner  spends  in  a  course,  the  system 
automatically  adapts  the  level  of  instruction  to  the  learner's  functional  level  based 
on  the  learner's  actual  performance  regardless  of  initial  enrollment  level. 
•Special  tutorial  support  in  the  form  of  focused  sequential  practice,  interactive 
tutorials,  repeated  prerequisite  instruction*  and  immediate  or  delayed  review  is 
activated  when  a  learner  has  difficulty  in  mastering  a  learning  objective. 

CCC  Adaptive  Instructional  Strategies 

CCC  courseware  attempts  to  improve  learning  conditions  through  individualized 
instruction  in  which  learning  activities  are  continuaK-y  >.^iapted  to  individual  learning 
differences.  The  CCC  courseware  used  for  this  study  z&  Math  Concepts  and  Skills.  The 
Teacher's  Handbook  for  Math  Concepts  and  Skills  (1991)  identifies  several  adaptive 
strategies  that  occur  in  this  program: 

( 1)        Motion  algorithms.  A  learner's  path  through  the  curriculum  is  determined  by  an 
evaluation  of  his  or  her  performance  by  the  automatic  management  system  as  the 
learner  works  each  exercise.  The  management  program  uses  this  evaluation  to 
select  the  next  exercise  or  tutorial.  Therefore,  each  learner  advances  through  a  course 
along  a  unique  path  that  is  shaped  dynamically  based  on  he  learner's  responses. 
The  selection  of  exercises  during  a  learner's  session  is  based  on  the  learner's 
performance  level  ir.  each  active  content  area  or  strand  and  the  sequence  of 
responses  the  learner  makes  to  the  exercises.  A  skill  remains  active  until  the 
learner  demonstrates  mastery  of  it  or  takes  the  maximum  number  of  exercises 
allowed  for  the  skill.  Figure  2  summarizes  the  decisions  the  CCC  management 
system  makes  while  a  learner  works  a  sequence  of  exercises  in  a  particular  skill. 
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  Evaluate  Learner  Performance 

AFTER  EACH  SESSION 


Current  _  YES 

_    Level    _  _  Begin  Next  Session 

_     OK?     _  at  Same  Level 


NO 


Difficulty.  YES 

at  Current  _  _  Subtract  ,20  of  a 

Level?    _           Grade  Year  from  Each 
 _  _  Strand  Level 


.NO 


Could  Work  _  YES 

at  Higher  _  _  Add  .20  of  a  Grade 

Level?    _           Year  to  Each  Strand 
 _  _  Level 


_End  of  Session. 


Figure  2,  Decision  process  used  to  determine  Initial  Placement  Motion  in  CCC's 
Math  Concepts  and  Skills.  

(2)  Initial  Placement  Motion  (1PM).  IPM  is  an  adaptive  process  that  successively 
revises  the  learner's  grade  level  in  each  strand  as  set  by  the  standard  course  motion.  Each 
learner  is  enrolled  in  the  course  based  on  performance  or  achievement  external  to  the 
program.  The  IPM  process  is  active  during  the  learner's  first  ten  complete  sessions  and 
evaluates  learner  performance  and  revises  the  learner  level  at  the  end  of  each  session  as 
shown  in  Figure  3.  The  assumption  is  made  by  the  CCC  management  program  that  the 
duration  of  IPM  and  the  magnitude  of  its  increases  are  such  that  the  process  provides  a 
comparable  measure  of  the  learner's  performance  and  stabilizes  on  the  learner's  functional 
level  in  most  cases. 

(3)  Proportion  of  exercises.  The  proportion  of  exercises  presented  from  each  strand  is 
based  on  a  fixed  proportion  and  an  individualized  proportion.  The  fixed  proportion 
weights  each  strand  relative  to  the  other  strands  and  is  derived  from  the  standard 
weighting  provided  in  most  elementary  mathematics  textbooks.  The  fixed  proportion 
sets  the  initial  probabilities  the 

management  system  follows  in  selecting  strands  and  is  modified  by  the 
management  system  for  an  individual  learner  based  on  his  or  her  performance  in  the 
strands.  When  a  learner  falls  behind  in  a  particular  strand,  the  proportion  of 
exercises  provided  to  a  learner  in  the  respective  strand  where  the  learner  is  below 
his  or  her  average  level  is  automatically  increased.  Adjustment  of  the  individualized 
proportion  is  independently  determined  for  each  learner. 
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-      evaluate  Lem  uel  rerionnauce 

AFTER  EACH  EXERCISE 


Learner  _  YES 

Performance  _-  

Excellent?  _  select  and  present  next 
 _  _  skill 


Pass  current  skill  and 


_  NO 


Present  another  exercise 


Learner  _  YES 

Performance  _  

Good?    _  from  current  skill  using 

.  _  _  mixed  presentation  format 

NO 


Present  another  exercise 


,  Learner  _  YES 

Performance  _  

Lower?    _           from  current  skill  using 
.  _  sequenced  practice  format 


Learner  _  YES 

Performance  _  _ 

Low?     _  skill  if  available 


Present  tutorial  for 


NO 


Several 
system  resources 
allow  learners  to 
control  the 
instructional 
process.  These 
resources  include 
Help,  Audio 
Repeat,  and  Tutor. 
Help  provides  an 
example  of  how  to 
answer  a 

particular  exercise. 
Audio  Repeat  is 
present  during  any 
session  that 
includes  exercises 
in  which  audio 
instructions  and 
reinforcement  are 
available.  General 
tutorials  provide 
examples  of  how  to 
work  a  particular 
type  of  exercise. 
Context-specific 
tutorials  explain  a 
specific  step  or 
operation  in  an 
exercise. 

CCC  Courseware, 
Content  and 

Structure 

According 
to  the  Teacher's 
Handbook  for 
Math  Concepts 
and  Skills  (1991) 
the  skills 
presented  by  the 
course  reflect  the 
content  of  current 
elementary 
mathematics. 

Course  content  includes  a  broad  range  of  higher-order  thinking  skills  to  complement  the 
basic  skills  taught  in  the  course.  Math  Concepts  and  Skills  is  organized  by  grade  level 
within  each  strand.  Each  objective  in  a  strand  includes  at  least  one  sample  exercise.  There 
are  Ui  content  areas  or  strands  and  a  total  of  1,136  mathematics  skills  that  are  typically 


Continued  _  YES 
Low  _  


Present  series  of 


_  Performance?. 


prerequisite  skills 


_NO 


_  Not  Reached  _  YES 

_  Mastery  _  

.after  20  ex.?_  immediately  or  later 

_ —  _  _  depending  on  skill 


-_  Mark  skill  for  review 


NO 


_  No  mastery-  _  YES 
_  after  all   _ —   


-_  Complete  skill  without 


instruction?. 


mastery  and  move  to  next 
 _  skill 


-_End  of  Exercise. 


Figure  3.  Decision  process  used  to  detei 


standard  motion  and  learner  progress  in  CCCs  Math  Concepts  and  Skills. 
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taught  in  kindergarten  through  the  end  of  the  eighth  grade.  The  CCC  management  program 
adds  or  drops  strands  as  learners  progress  through  various  grade  levels  of  the  course. 

The  Teacher's  Handbook  for  Math  Concepts  and  Skills  states  that  the  most  effective 
way  to  ensure  long-term  rentention  of  skills  is  a  "mixed  presentation  ...  in  which  the 
student  sees  a  mixture  of  exercises  from  all  the  strands  active  at  his  or  her  grade  level"  (p. 
14).  Course  sequencing  employs  a  mixed  presentation  strategy  by  selecting  exercises  from 
several  strands  during  each  session. 

Math  Concepts  and  Skills  uses  two  formats  for  presentation  of  math  exercises  on  the 
computer  screen.  Standard  arithmetic  formats  require  learners  to  respond  by  typing  words 
or  numbers.  Graphic  formats  use  graphic  representations  of  figures  and  objects  to  illustrate 
concepts  and  skills  and  learners  respond  by  clicking  the  mouse  pointer  on  objects  on  the 
screen,  moving  objects,  drawing  lines,  or  entering  numbers  or  words.  Both  exercise  types  are 
instructional,  reinforcing,  and  corrective. 

Hypotheses  and  Research  Objectives  for  Micro  Theory  Research 

There  is  evidence  in  research  related  to  learning  and  instruction  that  individual 
learners  require  different  levels  of  instruction  based  upon  their  aptitude  in  the  skills  being 
taught.  There  is  further  evidence  that  measures  of  prior  and  on-task  achievement  generally 
provide  strong  indicators  of  individual  learning  needs.  Computer-based  instructional 
systems  possess  the  potential  for  adapting  instructional  support  to  the  individual 
differences  and  aptitudes  among  learners. 

Rather  than  employ  classical  statistical  theories  that  compute  a  coefficient  based  on 
some  sort  of  group  average,  research  projects  that  examine  the  effects  of  individualized 
instructional  strategies  should  compute  a  coefficient  based  on  factors  that  characterize  a 
different  treatment  for  each  individual  learner  because  the  treatment  is  different  for  each 
learner.  This  paper  proposes  research  objectives  that  examine  the  effects  of  the  adaptive 
instructional  strategies  employed  by  the  CCC  integrated  learning  system  by  formulating  a 
predictive  equation  for  each  learner  about  expected  learner  outcomes  using  variables  derived 
from  the  individual  performance  of  each  learner.  This  predictive  equation  is  based  on  a 
theory  developed  by  Suppes,  Fletcher,  and  Zanotti  (1975,  1976)  in  which  the  amount  of  time 
a  learner  spends  on  a  learning  activity  is  a  function  of  the  learning  progress  made  by  the 
leaner.  A  learner's  achievement  as  related  to  the  course  objectives  is  expressed  as  post- 
treatment  grade  placement. 

Based  on  the  adaptive  processes  and  strategies  employed  by  CCC  courseware,  this 
paper  proposes  four  research  objectives  that  hypothesize  the  predictability  of  learner 
outcomes  as  a  result  of  the  adequacy  of  these  adaptive  mechanisms.  These  research 
objectives  are  relevant  for  validation  through  statistical  procedures: 

(1)  The  instructional  treatment  provided  by  the  CCC  integrated  learning  system  is 
sufficiently  adaptive  to  formulate  reliable  predictions  about  individual  learner 
achievement.  This  hypothesis  is  intended  to  test  the  degree  of  adaptation  of 
instruction  through  standard  motion  and  other  adaptive  strategies  by  determining  if 
there  is  a  significant  difference  between  predicted  learner  outcomes  and  actual 
learner  outcomes. 

(2)  The  CCC  integrated  learning  system  provides  a  more  reliable  estimate  of  learner 
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aptitude  than  an  external  pretask  measure  of  learner  achievement.  This  hypothesis 
is  intended  to  test  the  Initial  Placement  Motion  strategy  of  the  CCC  management 
system. 

(3)  Increased  instructional  time  and  adaptive  iterations  using  the  CCC  integrated 
learning  system  increase  the  reliability  of  the  prediction  of  learner  achievement. 
This  hypothesis  is  intended  to  determine  if  the  precision  of  the  predictive  equation  is 
improved  as  engaged  learning  time  is  increased. 

(4)  Differences  between  aptitude  and  achievement  among  learners  using  the  CCC 
integrated  learning  system  decrease  as  instructional  time  and  iterative  adaptations 
increase.  This  hypothesis  is  intended  to  test  the  assumption  that  an  effective  adaptive 
learning  strategy  induces  a  leveling  effect  or  flattens  differences  between  high  and  low 
aptitude  learners. 


Summary  and  Conclusion 

To  improve  the  effectiveness  of  their  product*,  ILS  vendor*  and  developers  must 
continually  analyze  how  to  revise  and  refine  their  cuursownre,  basing  it  on  a  morecomplete 
and  accurate  theory  of  effective  instruction  ami  lea  ruing.  Computer  Curriculum  Corporation, 
as  well  as  most  1LS  vendors,  claims  that  its  system  of  computer-based  instruction  provides 
significant  increases  in  learner  achievement  based  on  individualized  instruction  through 
adaptive  strategies  that  account  for  individual  learning  differences.  Research  must  be 
conducted  to  determine  the  success  or  failure  of  instructional  strategies  employed  by  ILSs 
and  validate  the  claims  of  ILS  vendors. 

The  basis  for  a  school  in  purchasing  technology  in  many  cases  is  the  improvement  of 
achievement  scores  in  all  areas  (Bender.  If>!>l),  Companies  that  market  ILSs  often  claim 
that  their  product  can  improve  scores  on  achievement  tests.  If  ILSs  are  proven  to  improve 
scores  on  achievement  tests,  the  next  questions  asked  are  by  how  much  and  what  is  the 
cost?  For  school  officials  price  is  obviously  a  major  consideration  when  purchasing 
educational  technology.  School  offic  ials  must  have  the  data  available  to  compare  the  relative 
advantages  of  ILSs  to  other  forms  of  instructional  technology  in  order  to  determine  which 
technology  provides  the  most  education  per  dollar. 

The  research  model  described  in  this  manuscript  is  intended  to  contribute  to  the 
body  of  knowledge  regarding  computer-based  instruction  as  well  as  propose  possible 
research  objectives  for  statistical  validation.  This  research  model  attempts  to  focus  on 
particular  design  variables  of  computer-based  instruction.  When  highly  individualized 
instructional  strategies  are  used  in  a  research  pro;  ct,  classical  research  designs  often  do  not 
produce  a  satisfactory  analysis  of  the  data.  The  research  paradigm  described  in  this 
manuscript  employs  a  micro  analysis  technique  based  on  a  regression  model  as  proposed  by 
Suppes  et  al.  (1976)  and  Malone  et  al.  (1979)  that  operationalizes  on  variables  for  each 
individual. 
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Despite  the  emphasis  on  cognitive  learning  principles  in  contemporary  instructional 
theories,  strategies  that  initiate  only  a  superficial  level  of  processing  continue  to  dominate 
computer-based  instruction  (CBI)  (Hooper  &  Hannafin,  1991;  Winn,  1988).  A  direct 
response  to  this  problem  is  to  make  greater  use  of  generative  strategies  in  CBI  designs 
(Jonassen,  1988).  Generative  learning  is  intended  to  promote  interpretation  and 
comprehension  by  requiring  learne*  s  to  relate  new  information  to  their  existing  knowledge 
(Jonassen,  1988).  Consequently,  the  level  of  processing  is  deepened  through  the  activation 
of  existing  knowledge  schemata  (Craik  &  Lockhart,  1972;  Wittrock,  1978,  1989). 

A  common  means  of  engendering  generative  processes  is  through  elaboration 
strategies  (Jonassen,  1988;  Mayer,  1980;  Wittrock,  1989).  Researchers  and  practitioners 
have  described  elaboration  strategies  in  several  ways,  including  cognitive  skills  that  add 
meaning  to  new  information  (Rohwer,  1970),  encoding  strategies  that  link  new  information 
with  existing  knowledge  (Brezin,  1980),  comprehension/retention  strategies  that  increase  the 
memorability  of  material  (Dansereau  et  al.,1979),  and  information-processing  strategies 
through  which  learners  make  the  material  more  meaningful  by  "adding  to"  or  "expanding  on" 
the  presented  information  (Jonassen,  1988). 

Uses  of  elaboration  strategies  have  generally  shown  positive  effects  on  learning, 
provided  the  elaborations  are  meaningful  to  learners  and  directly  relevant  to  instructional 
objectives  (see  Mayer,  1980).  Previous  studies  have  shown  positive  effects  on  achievement 
using  a  variety  of  verbal  and/or  imaginal  elaboration  strategies  with  diverse  learner  groups, 
including  elementary  school  students  (Barker,  1987),  secondary  students  (Weinstein,  1978b) 
undergraduates  (Dansereau  et  al.,  1979),  and  military  recruits  (McCombs  &  Dobrovonlny, 
1982).  In  cases  where  elaboration  has  not  proven  effective,  the  major  problems  inferred 
have  been  subjects'  inexperience  with  the  particular  elaboration  strategies  used  or  lack  of 
congruence  between  the  elaboration  activities  and  the  learning  outcomes  assessed  (e.g., 
Agesilas,  1987;  Weinstein,  Rood,  Roper,  Underwood,  &  Wicker,  1980). 

Concerns  about  student  abilities  to  generate  appropriate  elaborations  have 
prompted  efforts  to  examine  the  effects  of  subject-generated  as  opposed  to  experimenter- 
generated  elaborations.  Although  results  from  verbal  learning  studies  have  generally 
favored      ject-generated  elaborations  on  memory  tasks  (e,g.,  Bobrow  &  Bower,  1969; 
Slame  ..a  &  Graf,  1987),  concerns  have  been  expressed  regarding  how  effectively  students 
can  generate  elaborations  on  their  own  in  more  complex  learning  situations.  This 
consideration  raises  the  question  of  whether  elaboration  skills  training  could  be  used  to  help 
learners  improve  the  quality  of  their  elaborations  (Jonassen,  1988.)  The  most  common  form 
of  elaboration  skills  training  is  "detached  programs"  (Rigney,  1978)  which  operate 
independently  of  the  instructional  presentation.  Dansereau  et  al.  (1979),  used  this 
orientation  by  giving  undergraduates  two  hours  of  weekly  training  in 

comprehension/retention  techniques  over  a  12-week  period.  Results  from  that  study  as  well 
as  several  others  (e,g.,  McCombs  &  Dobrovonlny,  1982;  Pflaum,  Benton,  Glover,  &  Ronning, 
1980;  Weinstein,  1978a)  supported  the  training  condition  over  a  control  (no-training) 
condition. 

An  alternative  elaboration  training  method  is  the  embedded  approach  (Rigney, 
1978).  Jones  (1983)  describes  an  application  in  the  Chicago  Mastery  Learning  Reading 
Program  with  Learning  Strategies  (CMLR/LS).  The  program  incorporated  information 
processing  instructions  directly  into  reading  materials  used  by  teachers  and  students.  Step- 
by-stop  prompts*  multistep  directions  on  how  to  think,  adjunct  study  questions,  and  study 
prompts  are  examples  of  the  types  of  embedded  training  aids  used.  Unfortunately,  no 
formal  evaluation  of  the  CMLR/LS  program  was  conducted;  nor  have  embedded  strategies 
received  much  attention  in  the  literature.  Relative  to  detached  strategies,  the  embedded 
approach  has  the  advantage  of  making  the  training  contiguous  and  directly  integrated  with 
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Ilif  instructional  task  (hinder  &  Rickards,  1985;  O'Dell,  1990).  The  embedded  approach 
1 1 1  hi  i  HeeniH  highly  compatible  with  CBI  due  to  the  ease  of  making  appropriate  training 
frames  iiucl/ur  elaboration  prompting  standard  or  adaptive  features  of  the  inHtrucfinriJil 
program  Ah  the  learner  acquires  increased  experience  in  making  elaborations 
iiHlf«|ipiulontlyB  such  Kiipport  can  be  gradually  faded  and  eventually  eliminated. 

Tin*  purpose  of  the  present  study  was  to  extend  previous  research  by  examining  the 
relative  effects  far  adult  learners  of  receiving  experimenter-provided  elaborations  or 
generating  personal  elaborations,  using  strategies  taught  via  detached  versus  embedded 
training  Where  previous  studies  have  concentrated  primarily  on  the  effects  nf  elaboration 
on  rote  learning,  the  present  focus  included  higher-order  learning  as  well  (e.g.,  application  of 
new  information).  It  was  hypothesized  that  higher  achievement  would  be  associated  with 
uhcs  of:  (a)  learner-generated  and  experimenter-provided  elaboration  strategies  relative  to  a 
control  (no-elaboration)  condition,  (b)  learner-generated  relative  to  the  experimenter-provided 
elaborations,  and  (c)  embedded-  relative  to  detached-elaboration  training.  Additional 
research  interests  were  the  types  of  elaborations  generated  and  the  relationships  between 
types  of  subject  elaborations  and  learning  outcomes. 

Method 

Subjects  and  Design 

Subjects  were  80  employees  enrolled  in  a  professional  development  course  at  a  large 
corporation  in  a  midwestern  city.  The  course  content  dealt  with  diverse  topics  such  as 
improving  communications  with  managers  and  co  workers,  understanding  one's  own  job, 
interacting  with  others  to  improve  performance,  and  similar  topics  related  to  interpersonal 
communications.  The  subjects  were  all  administrative  assistants  with  extensive  typing  and 
computer  experience,  but  no  prior  experience  with  the  particular  course.  The  experimental 
treatments  were  integrated  with  the  regular  course  instruction  so  as  to  serve  as  the  actual 
lesson  material.  Subjects  were  recruited  by  sending  a  letter  to  350  corporate  employees 
asking  for  volunteers  for  a  training  study.  Employees  called  and  signed-up  for  a  convenient 
time  to  participate. 

The  First  80  subjects  who  volunteered  were  randomly  assigned  to  one  of  four  groups 
(il=20  in  each):  (a)  a  control  group  that  received  no  elaborations  or  elaborations  training,  (b) 
a  group  that  received  experimenter-provided  elaborations  during  the  lesson,  (c)  a  group  that 
received  detached  elaborations  training  and  generated  elaborations  during  the  lesson,  and 
(d)  a  group  that  received  embedded  elaborations  training  and  generated  elaborations  during 
the  lesson.  Major  dependent  variables  were  recall,  recognition,  and  application  of  the  lesson 
content  as  measured  by  a  posttest.  In  addition,  content  analyses  were  made  of  the 
elaborations  produced  in  the  two  learner-generated  elaboration  treatments.  An  alpha  level 
of  .OH  was  used  in  judging  significance  in  all  statistical  tests. 

Training  limgmm  and  Materials 

Elaboration  strategies  training  program.  A  one-hour  training  unit  on  elaboration 
strategies  was  designed  based  on  effective  programs  described  in  the  literature  (Dansereau 
et  til.,  1979;  Woinstein,  1978b;  Weinstein  et  al.,  1980).  The  unit  was  presented  to  the 
entire  group  in  one  aession  by  the  first  author,  The  introductory  section  emphasized  the 
need  for  instructional  training  that  stimulates  thinking  and  engages  participants  in  the 
learning  process.  Several  techniques,  such  as  mnemonic  devices  and  analogies,  were  then 
introduced  using  specific  examples  from  the  corporation's  existing  training  courses.  Part  2 
described  elaboration  in  terms  of  its  value,  features,  specific  techniques  (mnemonics, 
paraphrasing,  etc.),  and  effective  procedures,  Part  3  discussed  and  illustrated  uses  of 
elaboration  in  other  training  and  work  situations. 

Instructional  unit.  The  instructional  unit,  adapted  from  "Communicating  for 
Productivity"  (D'Aprix,  1982),  focused  on  a  model  of  supervisory  communication.  The  unit 
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contained  three  sections:  introduction  to  the  lesson,  new  instruction,  and  summary  of  main 
points.  The  new  instruction  section  covered  six  supervisory  tasks  and  specific 

TM 

communication  practices  for  each  task.  Using  HyperCard    ,  four  variations  of  the 

TM 

instructional  section  were  designed  for  use  on  a  Macintosh  computer. 

One  course  variation,  designed  for  the  control  group,  presented  primary  displays 
(major  content)  but  no  secondary  displays  (elaborations).  The  first  computer  frame 
presented  a  description  of  the  supervisory  principle  while  the  second  frame  presented  the 
specific  communication  activities  for  the  principle. 

A  second  variation  designed  for  the  experimenter-generated  elaborations  group, 
presented  primary  displays  (major  content)  as  well  as  secondary  displays  (elaborations). 
Experimenter-provided  verbal  elaborations  included  descriptive  information  that  related  the. 
content  to  current  job  experiences  and/or  stated  implications  for  use  of  the  supervisory 
model.  To  design  the  elaborations  as  precisely  as  possible,  a  company  subject  matter  expert 
assisted  in  the  development  of  the  elaborations. 

A  third  variation,  designed  for  the  learner-generated  elaborations  detached  training 
group,  presented  primary  displays  but  no  secondary  displays.  Instead,  orienting  tasks 
instructing  the  subject  to  generate  elaborations  after  each  primary  display  were 
incorporated.  The  learner  was  specifically  told  to  type  descriptive  information  relating  the 
content  to  current  job  experiences  and,  where  relevant,  stating  implications  of  the  concepts 
described  to  the  job  environment. 

A  fourth  version,  designed  for  the  learner-generated  elaborations,  embedded  training 
group,  contained  primary  displays  followed  by  an  elaboration  strategies  instructional  unit. 
The  unit  was  patterned  on  the  detached -training  program  described  in  the  preceding  section, 
with  appropriate  modifications  required  for  computer  presentation  and  immediate  on-line 
applications  of  the  strategies.  The  instruction  provided  an  overview  of  elaboration 
strategies,  descriptive  information  regarding  their  value  and  features,  and  an  overview  of 
specific  elaboration  techniques.  As  for  the  detached-training  treatment,  subsequent  lesson 
frames  included  orienting  directions  instructing  the  subject  to  generate  elaborations  relating 
the  content  of  each  primary  display  to  current  job  experiences.  An  on  line  help  system  for 
the  using  elaboration  strategies  was  incorporated.  The  help  system  was  an  electronic 
version  of  the  "effective  elaborating  procedures"  section  (Part  2)  of  the  detached  training  unit. 
The  learner-generated  elaborations  were  stored  on  disk  for  latter  analysis. 

The  four  lesson  formats  were  evaluated  by  one  corporate  and  two  academic 
instructional  designers,  and  were  field  tested  with  10  administrative  assistants  from  the 
same  population  as  participants  in  the  study.  Revisions  of  the  lesson  formats  were  made 
hased  on  the  review  and  developmental  testing. 

Achievement  test.  A  paper  and  pencil  achievement  test,  consisting  of  8  recall,  10 
recognition,  and  8  applications  items  was  designed.  Recall  questions  required  subjects  to 
name  the  supervisory  principles  and  specific  communication  activities.  For  example,  one 
item  was  "Name  the  principle  that  provides  the  employee  with  the  answer  to  the  question 
'How  am  I  doing?1"  Recognition  questions  listed  communication  activities  contained  in  the 
lesson,  and  asked  subjects  to  associate  each  with  one  of  the  six  principles  listed.  To 
measure  ability  to  apply  the  content,  a  case  study  describing  a  work  situation  was 
presented.  Multiple-choice  questions  required  subjects  to  resolve  different  case-related 
problem  situations  based  on  what  they  had  learned  from  the  lesson. 

The  achievement  test  was  evaluated  for  face  validity  by  three  instructional  designers 
and  was  pilot  tested  with  10  administrative  assistants  who  had  reviewed  the  lesson 
material  and  20  who  had  not  reviewed  it.  The  instrument  was  revised  as  suggested  by  the 
panel  review  and  by  the  item  analysis  (e.g.,  individual  items  that  were  correctly  answered 
by  7f)%  or  more  of  the  pilot  test  sample  were  replaced  or  increased  in  difficulty).  Internal- 


consistency  reliability  coefficients  for  the  final  test,  computed  via  Cronbach's  Alpha  formula, 
ranged  from  a  low  value  of  .82  for  the  recognition  section  to  a  high  of  .92  for  the  recall 
section. 
Procedure. 

One  week  prior  to  the  initiation  of  the  instructional  phase,  subjects  in  the  detached- 
training  group  attended  the  elaboration  strategies  training  session  in  the  corporation's 
training  facility.  The  instructional  phase  was  implemented  during  a  normal  work  week  as 
done  for  most  training  courses  at  the  corporate  headquarters.  Subjects  reported  to  the 
assigned  training  room  in  groups  of  five,  one  per  computer.  Before  beginning  the  lesson,  the 
proctor  read  instructions  stating  that  (a)  participants  should  try  to  do  their  best  in  learning 
the  new  information;  (b)  there  would  be  a  test  on  the  material;  and  (c)  there  would  be  no 
time  limits.  Subjects  in  the  two  elaboration  conditions  were  also  told  that  spelling, 
punctuation,  capitalization,  and  typing  mechanics  could  be  disregarded;  and  there  was  no 
need  to  spend  time  on  formatting  text.  The  proctor  remained  in  the  room  to  provide 
assistance  when  needed.  After  each  subject  completed  the  assigned  treatment,  he/she  took 
a  10-minute  break  and  then  returned  for  administration  of  the  posttest. 

Results 

Achievement 

Posttest  scores  were  analyzed  via  a  one-way  multivariate  analysis  of  variance  in 
which  the  dependent  variables  were  recall,  recognition,  and  application.  Treatment  means 
and  standard  deviations  on  these  variables,  along  with  total  score  (maximum  points=26), 
are  shown  in  Table  1.  As  shown  in  the  table,  the  direction  of  the  means  on  all  subtests  and 
on  the  total  score  favor  the  two  learner-generated  elaborations  groups  (detached  and 
embedded)  over  the  control  group  and  the  experimenter-provided  elaborations  group. 
Statistical  results  showed  the  overall  multivariate  treatment  effect  to  be  significant. 
Univariate  analysis  of  variance  indicated  significant  treatment  effects  on  recall  scores,  E  (3, 
76)  =  7.90,  MSW  =  2.12;  and  application  scores,  £  (3,  76)  =  5.41,  MSW  =  1.47;  but  not  on 
recognition  scores. 


Insert  Table  1  about  here 


Tukey  pairwise  comparisons  of  the  recall  means  indicated  that  the  performance  of 
both  the  detached-training  group  (M  =  6.80)  and  the  embedded-training  group  (M=  7.35) 
surpassed  the  performance  of  the  control  group  (M  =  5.20).  The  experimenter-provided 
elaborations  group  (M  =  6.30)  did  not  differ  significantly  from  any  of  the  groups. 

Results  of  Tukey  followups  on  the  application  means  indicated  that  the  embedded- 
training  group  (M  =  5.50)  surpassed  the  control  group  (M  =  4.00).  The  experimenter- 
provided  elaborations  group  (M  =  4,65)  and  the  detached-training  group  (M  =  5.00)  did  not 
significantly  differ  from  any  of  the  groups. 
Analysis  of  Elaborations 

The  elaborations  generated  by  subjects  in  the  embedded-  and  detached-training 
groups  (total  n  per  individual  =  6)  were  analyzed  initially  via  a  total  word  count  to 
determine  verbalization  length.  Although  the  embedded-training  group  gave  slightly  longer 
elaborations  (M  =  477.45  words)  than  did  the  detached-training  group  (M  =  446.45  words), 
a  t-test  comparison  of  these  means  was  not  significant.  Both  group  means,  however,  were 
significantly  lower  than  the  word  count  of  659  for  the  experimenter-provided  elaborations. 
Word  counts  were  not  significantly  correlated  with  recall,  recognition,  application,  or  total 
scores  in  any  elaboration  treatment  or  for  all  elaboration  subjects  combined. 
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Each  elaboration  was  further  analyzed  and  categorized  in  two  ways.  One  dimension 
was  "personal"  vs  "impersonal."  Elaborations  that  contained  personal  nouns,  specific 
names  of  people  and  departments,  or  examples  of  private  or  personal  situations  were 
classified  as  "personal."  The  following  elaboration  given  by  one  subject  exemplifies  this 
category: 

In  my  group,  I  have  yet  to  hear  what  our  goals  and  mission  are.  I  have  not  had 
a  review  with  my  supervisor  or  manager  and  have  asked  for  one.  I  feel  like  I'm 
doing  a  good  job,  but  no  one  has  confirmed  this.  I  have  to  prompt  my  supervisor 
for  feedback  and  receive  very  little  guidance.  I  feel  out  of  place  in  our  department 
because  I  am  not  sure  what  all  the  services  are  that  we  offer.  I  have  been  there 
for  9  months  and  feel  an  overview  of  the  department  would  have  been  helpful. 
Luckily,  I  am  the  type  of  person  that  finds  out  for  myself.  I  feel  our  department 
needs  a  lot  of  help. 

On  the  other  hand,  elaborations  that  contained  second-person  language  (you,  your), 
generic  names  of  people  and  departments  (employees,  managers,  etc.),  and  company-wide 
policies  or  procedures  for  examples  were  classified  as  "impersonal."  The  following 
elaboration  represents  this  classification: 

By  clarifying  work  unit  goals  and  objective,  the  employees  are  better  informed  of 
the  mission,  goals,  and  objectives  of  the  department.  Without  the  knowledge, 
the  employees  would  be  going  off  in  different  directions.  Having  a  manager  keep 
you  current  on  the  department's  goals  and  objectives  helps  you  do  a  better  job 
and  be  more  productive.  Also,  by  keeping  your  employees  informed,  they  feel 
more  like  part  of  the  team  and  their  contributions  are  valued. 
Second,  each  elaboration  was  classified  in  relation  to  the  lesson  content  as 
"irrelevant,"  "paraphrased,"  or  "new  idea."  If  the  elaboration  was  obviously  not  related  or 
incidental  to  the  content,  it  was  categorized  as  "irrelevant."  If  it  was  a  restatement  of  the 
principles  and  activities  presented  in  the  lesson,  it  was  categorized  as  "paraphrased." 
When  the  elaboration  expanded  on  the  information  presented  so  that  new  concepts, 
principles,  and/or  activities  emerged,  it  was  categorized  as  "new  idea."  Classifications  were 
made  independently  by  two  raters.  Where  disagreements  occurred,  the  classifications  were 
discussed  and  revised  to  achieve  consensus. 

Inspection  of  the  descriptive  data  indicated  a  tendency  for  subjects  to  generate 
personal  elaborations  more  often  than  impersonal  elaborations  (71  percent  and  29  percent 
respectively),  C2  (1)=  40.83;  and  to  paraphrase  content  (60  percent)  more  often  than  they 
generated  new  ideas  (33  percent)  or  gave  irrelevant  elaborations  (7  percent)  C%  (2)=104.25. 

A  2(embedded  vs.  detached-training  group)  x  2(type  of  elaboration)  chi-square 
analysis  of  the  frequencies  of  personal  and  impersonal  elaborations  was  conducted  to 
determine  the  relationship  between  elaboration  training  and  type  of  elaboration  generated. 
Overall,  subjects  in  the  embedded-training  group  generated  a  significantly  greater  number  of 
personal  elaborations  (83  percent)  than  did  those  in  the  detached-training  group  (58 
percent),  C%  (1)  =  5ft.  68.  A  2(group)  x  3(type  of  elaboration),  chi-square  was  then  conducted 
on  the  number  of  irrelevant,  paraphrased,  and  new  idea  elaborations  made  by  subjects  in 
the  two  elaboration  groups.  It  yielded  a  significant  value  of  C2  (2)  =  109.02.  The  embedded 
training  group  paraphrased  the  content  more  frequently  (67%)  and  gave  new  ideas  less 
frequently  (2H%)  than  did  the  detached  training  group  (54%  and  38%,  respectively).  In  a 
final  series  of  analyses,  the  f  requency  counts  for  the  different  elaboration  categories  wore 
each  correlated  with  recall,  recognition,  application,  and  total  posttest  scores.  None  of  the 
relationships  was  significant. 


Discussion 

It  was  hypothesized  that  subjects  either  generating  elaborations  or  receiving 
experimenter-generated  elaborations  would  learn  more  effectively  than  would  a  control  gnrtp 
that  received  no  elaborations  or  instructions  to  elaborate.  The  results  supported  this 
hypothesis  for  subject-generated  but  not  for  experimenter-generated  elaborations. 

Consistent  with  the  findings  of  Mayer  (1980),  Barker  (1987),  and  Speaker  (1987), 
subjects  generating  elaborations  surpassed  control  group  subjects,  on  three  out  of  four 
posttest  comparisons.  The  generally  rich  and  detailed  elaborations  (averaging  close  to  100 
words  each)  of  the  two  learner-generated  groups  are  evidence  of  subjects  attempting  to  form 
relations  between  the  content  and  their  own  experiences.  While  the  majority  of  elaborations 
(60%)  were  paraphrases  of  the  content,  two- thirds  were  new  ideas  generated  from  the 
content.  Further,  almost  three-fourths  were  examples  from  subjects'  experiences,  thus 
including  personalization  as  a  dimension  to  increase  the  memorability  of  information 
(Kulhavy,  in  press)  and  its  relatability  to  existing  schemata  (Anand  &  Ross,  1987;  Davis- 
Dorsey,  Ross,  &  Morrison,  1991).  Only  on  the  recognition  measure,  which  was  least 
cognitively  demanding  of  the  three  subtests,  did  an  advantage  for  learner-generated 
elaborations  fail  to  occur. 

However,  in  contrast  to  other  studies  (Bobrow  and  Bower,  1969;  Slamecka  and  Graf, 
1978),  there  were  no  significant  advantages  for  either  learner-generated  elaborations  group 
relative  to  the  experimenter-generated  elaborations  group.  However,  results  on  the  all 
posttests  directionally  favored  the  learner-generated  elaborations  groups,  and,  as  just 
described,  the  latter  groups  significantly  surpassed  the  control  group  on  two  out  of  three 
subtests  where  the  experimenter-generated  condition  failed  to  do  so  on  any. 

In  contrast  to  the  verbal  learning  tasks  employed  in  previous  studies  on 
experimenter-provided  vs.  subject-generated  elaborations  (e.g.,  Bobrow  &  Bower,  1969),  the 
experimenter-provided  communication  principles  and  activities  used  here  directly  related  to 
subjects'  jobs  and  common  events  that  most  individuals  should  have  found  easy  to  relate  to 
everyday  work  experiences.  Several  open-ended  attitude  responses  suggested  covert 
attempts  to  generate  relations  between  the  experimenter  elaborations  and  individual 
experiences  1.  For  example,  one  subject  wrote: 

Some  managers,  like  my  own,  really  don't  have  the  qualifications  to  he  a  manager. 

She  doesn't  give  you  any  feedback,  unless  you've  really  done  something  terrible,  and 

she  doesn't  know  the  organization  of  the  company.  We  need  an  effective  training 

program  for  all  managers.  This  course  made  me  think  about  how  to  better 

communicate  with  my  manager,  and  what  steps  I  can  take  as  an  individual  to  get  the 

results  I  think  are  due  me  as  an  employee. 
It  is  thus  possible  that  the  level  of  generative  processing  achieved  in  the  experimenter- 
provided  group  was  sufficient  for  those  elaborations  to  serve  similar  functions  as  the  learner- 
generated  ones.  However,  higher-order  learning,  such  as  application  and  synthesis  of 
information,  is  difficult  to  measure,  particularly  using  instruction  of  short  duration  and 
objective-type  test  items.  Limitations  of  the  present  instruments  for  assessing  such  outcomes 
might  have  masked  stronger  treatment  differences  reflecting  deeper  processing  by  the  learner- 
generated  group. 

It  was  also  hypothesized  that  the  embedded  training  group  would  demonstrate  higher 
achievement  than  would  the  detached  training  approach.  This  hypothesis  was  not  supported 
by  posttest.  comparisons.  However,  from  an  efficiency  standpoint,  the  embedded  strategy  had 
the  advantage  of  integrating  the  delivery  of  the  elaboration  training  and  the  lesson  in  the 
same  presentation.  Accordingly,  subjects  could  proceed  through  the  training  at  their  own  pace 
and,  in  future  lessons,  bypass  the  training  if  it  were  no  longer  needed.  The  savings  from 
eliminating  the  detached  session  would  far  exceed  over  time  the  development  costs  of  the 


5  5  » 


elaboration  unit  (which,  with  minor  modifications,  could  he  easily  used  on  other  CBI  units  as 
well).  The  detached  training  also  seemed  more  successful  in  engendering  attempts  to  relate 
the  material  to  personal  experiences. 

From  an  instructional  design  standpoint,  the  results  overall  are  supportive  of 
incorporating  elaboration  strategies,  where  reasonable  and  feasible,  in  CBI  lessons.  In 
particular,  two  major  advantages  are  indicated  for  the  embedded  strategies  approach.  First, 
the  time  required  to  develop  the  embedded  training  and  orienting  tasks  for  the  learner- 
generated  elaborations  was  considerably  less  than  the  time  required  to  develop  the 
experimenter-provided  elaborations.  Second,  with  instructional  time  at  a  premium  in  both 
school  and  corporate  training  environments,  the  elimination  of  the  pretraining  session  by  the 
embedded  approach  can  substantially  increase  its  efficiency  relative  to  the  detached  training 
strategies. 

Another  consideration  for  designer*  is  learners'  ability  to  generate  verbal  elaborations 
from  the  content  presented.  In  fact,  all      jects  attempted  to  construct  some  type  of 
elaboration  at  each  prompt  to  do  so.  Of  the  240  (40  subjects  x  6)  elaborations  generated,  only 
16  (6.7%)  were  classified  as  "irrelevant,"  In  the  present  study,  subjects  generally  found  it  easy 
to  relate  the  material,  which  dealt  with  a  work-related  theme,  to  their  everyday  experiences. 
This  personalization  and  higher  level  of  learning  is  in  agreement  with  the  outcomes  predicted 
by  constructivists  when  using  a  generative  learning  approach  (Biggs,  1987;  Duffy  &  Jonassen, 
1991).  By  comparison,  Mayer  (1980)  presented  material  on  computer  programming  that  was 
less  familiar  to  subjects,  and  therefore  needed  to  be  accompanied  by  special  prompting  and 
illustrations  to  guide  subjects'  production  of  elaborations.  Effective  design  of  elaboration 
strategies  therefore  requires  careful  front-end  analyses  of  the  instructional  content  in  relation 
to  learner  backgrounds. 

Further  research  that  emphasizes  qualitative  analyses  of  elaborations  is  needed  to 
obtain  additional  insight  into  how  types  of  elaborations  influence  different  levels  of  learning. 
The  present  results  were  inconclusive  in  this  regard,  perhaps  as  a  result  of  insensitivity  of  the 
dependent  measures.  If  such  effects  are  known,  researchers  could  explore  uses  of  embedded 
strategies  training  to  guide  learners  on-line  in  generating  the  specific  types  of  elaboration  most 
suited  to  particular  learning  objectives. 


The  full  version  of  the  paper  was  published  in  1992,  Contemporary  Educational  Psychology, 
17,  126- 1»5. 


References 


Agesilas,  K.A.  (1987).  The  effect  of  meaningful  elaborations  and  summaries  on  the  UumuiAUg 
and  retention  of  prose  materials.  Unpublished  doctoral  dissertation,  University  of 
Tennessee. 

Anand,  IV,  &  Ross,  S.M.  (1987).  Using  computer-assisted  instruction  to  personalize 

arithmetic  materials  for  elementary  school  students.  Journal  of  Educational  Psychology. 
IS,  72-77. 

Barker,  J.tt.  (1987).  lnteL'ratinti  visual  and  verbal  elaboration  learning  into  the  elementary 
school  ctirriculupi:  A  model  of  haste  constructs.  Unpublished  doctoral  dissertation, 
University  of  Nevada. 

Biggs,  J.  (1987).  Student  approaches  to  learning  and  studying.  Melbourne:  Australian 

Council  for  Educational  Research. 
Bobrow,  S.A.,  &  Bower,  (  J.  11.  (1969).  Comprehension  and  recall  of  sentences.  Journal  of 

Experimental  Psychology.  80.  455-461. 
Brezin,  M.J.  (1980).  Cognitive  monitoring:  From  learning  theory  to  instructional 

applications.  Educational  Communications  and  Technology  Journal.  22*  227*242. 
Oaik,  K.I.M.,  Lockhart,  R.S.  (1972).  Levels  of  processing:  A  framework  for  memory 

research.  Journal  of  Verbal  Learninp  and  Verbal  Behavior.  11.  671-684. 
D'Aprix,  R.  (1982).  Communicating  for  productivity.  New  York:  Harper&  Row. 
Dansereau,  D.F.,  Collins,  K.W.,  McDonald,  B.A.,  liolley,  CD.,  Garland,  J.C.,  Diekhoff, 

O.M.,  &  Kvans,  S.H.  (1979).  Development  and  evaluation  of  an  effective  learning 

strategy  program.  Journal  of  Educational  Psychology.  79.  64-73. 
Davis-Dorsey,  J.D.,  Ross,  S.M.,  Morrison,  (i.R.  (1991).  The  role  of  rewording  and  context 

personalization  in  the  solving  of  mathematical  word  problems.  Journal  of  Educational 

Psychology,  52,61-68. 

Duffy,  T.  ML  &  Jonassen,  D.  H.  (1991).  Constructivism:  New  Implications  for  Instructional 

Technology.  Educational  Technology.  31.  7-12. 
Hooper,  S.,  &  Hannafin,  M.J.  (1991).  Psychological  perspectives  on  emerging  instructional 

techniques:  A  critical  analysis.  Educational  Psychologist  26.  69-95. 
Jonassen,  D.H.  (1988).  Integrating  learning  strategies  into  course  to  facilitate  deeper 

processing.  In  D.H.  Jonassen  (Ed.),  Instructional  designs  for  microcomputer  courseware 

(pp.  151.181).  Hillsdale,  NJ:  Lawrence  Erlbaum  Associates. 
Jones,  B.F.  (1983).  Integrating  learning  strategies  and  text  research  to  teach  high  order 

thinking  skills  in  schools.  Paper  presented  at  the  annual  meeting  of  the  American 

Educational  Research  Association,  Montreal,  Canada. 
Linder,  R.  W.  &  Rickards,  J.  (1985).  Questions  inserted  in  text:  Issues  and  implications.  In 

D.  Jonassen  (Ed).  The  technology  of  text:  Principles  for  structuring,  designing,  and 

displaying  text  volume  2. 
Mayer,  R.E.  (1980).  Elaboration  techniques  that  increase  the  meaningfulness  of  technical 

text:  An  experimental  test  of  the  learning  strategy  hypothesis.  Journal  of  Educational 

Psychology,  TL  770-784. 

McCombs,  B,L„  &  Dobrovonlny,  J.L.  (1982).  Student  motivational  skill  training  package: 
Evaluation  for  Air  Force  technical  training  (AFHRL-TP-82-31).  Lowry  Air  Force  UnH<\ 
CO;  Logistics  and  Technical  Training  Division. 

O'Dull,  J.  K.  (1990).  The  effects  of  varying  step  size  and  learner  response  activity  in  worked 
example  presentations  in  individual  learners'  ability  to  solve  mathematical  atorv 
problems.  Unpublished  doctoral  dissertation,  Memphis  State  University. 


Pflaum,  S.  W.,  Benton,  S.  LM  Glover,  J.  A.,  &  Ronning,  R.  R.  (1983).  Elaboration  and  recall 
of  main  ideas  in  prose.  Journal  of  Educational  Psychology.  75.  898-907. 

Rigney,  J.  W.  (1978).  Learning  Strategies:  A  theoretical  perspective.  In  H.  F.  O'Neil  (Ed.) 
Learning  Strategies  (pp.  165-203).  New  York:  Academic  Press. 

Rohwer,  W.D.  (1970).  Images  and  pictures  in  children's  learning.  Psychological  Bulletin. 
iii,  393-403. 

Slamecka,  N.J.,  &  Graf,  P.  (1978).  The  generation  effect:  Delineation  of  a  phenomenon. 

Journal  of  Experimental  Psychology:  Human  learning  and  memory.  4,  592-604. 
Speaker,  R.  B.  (1987).  Elaboration  and  prediction  in  the  processing  and  retention  of 

elementary  science  texts.  Unpublished  doctoral  dissertation,  University  of  California. 
Weinstein,  C.  E.  (1978a).  Learning  of  elaboration  strategies.  Unpublished  doctoral 

dissertation,  University  of  Texas. 
Weinstein,  C.  E.  (1978b).  Learning  of  elaboration  strategies.  In  H.  F.  O'Neil  (Ed.)  Learning 

Strategies  (pp.  31-55).  New  York:  Academic  Press. 
Weinstein,  C.  E.,  Rocd,  M.  M.,  Roper,  C,  Underwood,  V.L.,  &  Wicker,  F.  W.  (1980).  Field 

test  of  a  revised  form  of  the  cognitive  learning  strategies  training  program  with  armv 

enlisted  personnel  (Technical  Report  No.  462).  Austin,  TX:  U.S.  Army  Research 

Institute  for  the  Behavioral  and  Social  Sciences. 
Winn,  W.  (1988).  Toward  a  rationale  and  theoretical  basis  for  educational  technology. 

Educational  Technology.  22,  35-46. 
Witt  rock,  M.C.  (1978).  The  cognitive  movement  in  instruction.  Educational  Psychologist. 

15,  15-29. 

Wittrock,  M.C.  (1989).  Generative  processes  of  comprehension.  Educational  Psychologist. 
21,  345-376. 


556 


Table  1 

Means  and  Standard  Deviations  for  Achimmem  Subtests 


Achievement  Subtests 


Group  Recall  Recognition      Application  Total 


M 

SD 

M 

SD 

M 

SD 

M 

SD 

Control 

5.20 

2.29 

6.45 

1.76 

4.00 

0.97 

15.65 

3.54 

Experimenter -provided 

6.30 

1.17 

6.25 

1.83 

4.65 

1.42 

17.20 

3.07 

Learner-gen.,  detached 

6.80 

1.15 

6.80 

1.15 

5.00 

1.56 

18.60 

2.56 

Learner-gen.,  embedded 

7.35 

0.75 

6.55 

1.61 

5.50 

0.69 

19.40 

2.19 

Note:  n  =  20  in  each  group.  Scores  could  range  from  0  to  8  for  recall,  0  to  10  for 
recognition,  0  to  8  for  application,  and  0  to  26  for  total  score. 


Footnotes 

1  Attitude  data  comparing  treatment  groups'  reactions  to  the  lesson  and  to  the  elaboration  strateg 
where  used,  were  also  collected  huf  are  not  reported  in  this  article. 
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INTRODUCTION 
Purpose  of  the  study 

The  purpose  of  this  study  was  to  examine  how  Sunny  Days  Elementary  School  (a 
pseudo  name)  initiated  school  restructuring  using  Dr.  Charles  Reigeluth's  School 
Restructuring  Process  Model.  The  purpose  of  this  study  aimed  to  examine  how  the  actual 
experiences  of  Sunny  Days  Elementary  School  could  be  used  to  refine  and  improve  Dr. 
Reigeluth's  process  model. 

Background 

School  Restructuring. 

Our  rapidly  changing  economy,  environment  and  society  are  changing  the 
expectations  placed  on  public  schools  (Reigeluth,  1992).  Public  schools  are  being  asked  to  do 
mor2  with  less  resources  than  ever  before  (Sizer,  1989  &  (iovernors  Task  Force).  In  order  to 
meet  these  new  and  changing  challenges,  schools  are  turning  to  restructuring  as  a  way  of 
changing  how  they  deliver  educational  services.  dim  Oglesby  indicates  we  are  actually 
asking  public  schools  to  "re-invent  education"  (Oglesby,  199U). 

Unfortunately,  our  schools  have  little*  experience  in  restructuring  or  "re  inventing 
education".  Many  schools  are  finding  themselves  unprepared  to  implement  the  types  of 
changes  restructuring  requires.  Schools  are  increasingly  turning  to  models  for  assistance 
and  guidance  for  their  restructuring  efforts.  Dr.  Reigeluth's  process  model  was  designed  to 
offer  such  guidance. 

Reigeluth's  Process  Model 

Reigeluth's  model  recommends  school  restructuring  follow  a  three  phase  process. 
These  phases  are: 
Initiation/Preparation  Phasic 

1.  Assessing  the  readiness  of  the  community. 

2.  Get  an  outside  facilitator. 

3.  Get  commitment  from  all  stakeholder  groups. 

4.  Select  an  approach  for  the  change  effort. 

5.  Select  participants  for  the  coordinating  council  and  design  team. 

6.  Prepare  the  participants. 

7.  Relate  with  mm  participants. 

Design/Development  Phawe 

8.  Find  common  values  and  analyze  learner  and  societal  needs. 

9.  Develop  core  ideas  and  goals. 

If).       Develop  an  image,  and  design  n  systems  of  functions. 

1 1 .  Design  enabling  systems. 

12,  Analyze1  the  feasibility. 

Implement  at  loii/DoeunuMitation  Phase 

I M        Plan  the  implementation. 
1  I        Implement  the  design. 
1  <•>        htM  uiueiit  the  system. 

Although  the  activities  in  each  phase  are  not  considered  linear  or  independent  of  one 
another,  the  model  does  suggest  tin1  phases  build  on 
one  n  not  her 
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The  findings  of  this  study  are  intended  to  generalize  beyond  Sunny  Days  Elementary 
School  to  the  process  model  itself.  Formative  evaluation  has  proven  very  successful  in 
improving  materials,  in  this  case  it  was  applied  to  a  process  model.  By  using  this  case  as  a 
means  to  formatively  improve  a  much  needed  process  model,  future  restructuring  efforts  will 
be  benefit  from  a  better  model  to  guide  them  as  they  attempt  to  initiate  school  restructuring. 

This  study  focused  on  the  initiating  phase  of  school  restructuring  and  sought  to 
understand  how  Sunny  Days  Elementary  School  initiated  their  restructuring  process. 
Factors  that  influenced  Sunny  Days'  efforts  in  starting  the  process  were  identified  and 
analyzed.  Recommendations  for  modifying  Reigeluth's  process  model  to  better  serve  Sunny 
Day  s  Elementary  School  are  presented.  The  following  research  questions  guided  this  study. 

Research  Questions 

This  study  investigated  the  following  questions: 

1 .  How  does  what  they  are  experiencing  match  Reigeluth's  model? 

2 .  Does  Reigeluth's  model  represent  the  significant  events  of  their  experiences  during 
the  initial  phase  of  restructuring  ? 

3.  Are  other  events  occurring  during  this  phase  that  are  not  represented  by  the  model? 

4.  What  obstacles  are  they  encountering  and  what  inferences  can  be  made  about  they 
process  form  these  obstacles? 

5.  What  strides  are  they  making  and  what  inferences  can  be  made  to  the  process 
concerning  these  strides? 

METHODS 

Research  Design 

The  need  to  explore  school  restructuring  in  the  complex  setting  of  actual  public 
schools  is  important  to  understand  the  rich  interaction  of  the  many  factors  involved.  A  case 
study  methodology  was  selected  for  this  study  due  to  its  ability  to  examine  the  dynamic 
processes  of  school  restructuring.  Formative  research  methods  were  used  in  order  to  identify 
possible  improvements  to  Reigeluth's  process  model  based  on  insights  generated  by  the  case 
study. 

The  researcher  served  as  a  process  facilitator  at  bi-monthly  meetings  of  the 
Renovation  Committee  at  Sunny  Days  Elementary  School.  The  Renovation  Committee  was 
a  group  of  six  teachers  and  six  parents  that  volunteered  to  serve  in  an  advisory  role  for  the 
.school  restructuring  the  renovation  of  the  school  building.  Their  bi-monthly  meetings  were 
video  taped.  The  video  tapes  were  analyzed  to  identify  themes  and  patterns  of  interactions 
which  were  used  to  guide  and  focus  the  formal  and  informal  interviews. 

The  researcher  collected  and  analyzed  documents  created  hy  the  staff  of  Sunny  Days, 
including  materials  developed  in  their  committee  meetings.  Written  communications  to 
parents  and  community  members,  such  as,  news  paper  articles,  notes  sent  home  with 
students,  etc.,  were  collected  and  analyzed. 

Informal  interviews  were  conducted  and  focused  on  how  the  school  came  to  be 
involved  in  this  process,  what  other  options  they  considered  and  how  they  see  the  process 
progressing.  These  interviews  often  took  place  before  and  immediately  after  the  committee 
meetings. 

Committee  members  were  also  interviewed  using  more  formal  interview  techniques. 
The  interviews  sought  to  validate  t  he  accuracy  of  the  researcher's  observations  and  gather 
addition  information  about  what  they  were  experiencing  and  how  they  were  making  sense  of 
what  was  happening.  The  interviews  asked  participants  to  reflect  on  the  process  of 
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restructuring  and  offer  suggestions  for  improving  the  restructuring  process,  These  suggestion 
were  used  to  identify  possible  refinements  to  Reigeluth's  process  model. 

Sample 

Sunny  Days  Elementary  School  (a  pseudo  name)  was  selected  for  this  study  because 
of  their  interest  in  Reigeluth's  model  and  their  acceptance  of  the  theories  supporting  the 
model. 

Sunny  Days  Elementary  School  is  located  in  central  Indiana.  It  houses  grades  k-5. 
A  faculty  of  14  full  time  teachers,  2  part-time  teachers,  one  principal,  one  secretary  and  two 
custodians  provide  educational  services  to  approximately  340  students  at  Sunny  Days 
Elementary  School. 

Students  at  Sunny  Days  come  from  a  rural  area  that  consists  of  98%  white,  English 
speaking  families.  In  the  past,  agriculture  was  a  major  industry  in  the  area.  Like  most 
midwestern  rural  areas,  agriculture  is  providing  fewer  jobs  than  in  the  past.  An  increasing 
number  of  residence  are  commuting  to  the  metropolitan  city  30  miles  away  for  employment. 
The  area  has  unemployment  rates  typical  of  many  midwestern,  rural  communities. 
However,  their  unemployment  and  poverty  rates  are  lower  than  the  national  averages,  but 
greater  than  the  cities  around  them. 

The  school  consists  of  one  building  which  was  constructed  in  the  early  1960s.  This 
30+  year  old  building  no  longer  meets  building  codes  or  the  needs  of  students,  faculty  and 
staff.  Opportunities  to  utilize  technology  is  greatly  limited  by  the  current  condition  of  the 
school  building.  Faced  with  a  major  renovation  project,  the  principal  and  superintendent 
chose  this  opportunity  to  re-examine  how  educational  services  are  delivered  in  Sunny  Days 
Elementary  School. 

Sunny  Days  is  one  of  four  elementary  schools  in  their  district.  Two  elementary 
schools  and  two  secondary  schools  in  their  district  have  received  sizable  state  and  federal 
grants  for  implementing  educational  innovations.  These  schools  and  the  school  district  as  a 
whole,  are  considered  innovative  and  progressive  by  local  and  state  educational  agencies 
(Dave  Wilkinson,  Indiana  Department  Of  Education,  1992).  Sunny  Days  is  one  of  the  last 
schools  in  the  district  to  remodel  and  to  make  fundamental  changes  in  how  they  deliver 
educational  services. 


FINDINGS 

1.  Leadership 

The  words  and  actions  of  the  principal  were  not  congruent.  The  principals  take 
charge  actions  were  in  opposition  with  her  desire  to  create  an  empowered  staff.  The 
principal  continued  to  make  decisions  and  control  activities  the  committee  could  have 
handled.  They  were  told  to  take  charge  and  make  decisions,  but  were  not  allowed  to  do  so, 

2.  Power  of  the  driving  force 

The  renovation  of  the  building  provided  a  powerful  driving  force  for  the  restructuring 
effort.  Unfortunately,  at  times  it  was  easier  and  more  interesting  to  focus  on  paint  and 
carpet  colors  than  on  the  future  direction 
of  education, 

3.  Time 

The  amount  of  time  available  was  a  problem.  The  volunteers  were  meeting  on  their 
own  time  after  work.  This  also  meant  the  quality  of 
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the  time  was  diminished.  The  participants  were  tired  and  reported  not  having  the  energy 
they  would  have  liked  to  have  had  to  devote  to  this  project, 

4.  Short  term,  planning  focus 

While  this  group  stated  they  were  interested  in  long  term  improvement,  they 
continued  to  focus  on  short  term  quick  fixes.  The  complicated  solutions  they  considered  were 
often  closely  related  to  past  experiences,  many  of  which  had  not  been  successful.  They 
appeared  to  want  results  that  were  different  from  those  in  their  past,  but  were  hesitant  to 
try  new  methods.  Primarily,  their  focus  remained  on  individual  learning  and  what  they 
themselves  could  do  in  the  privacy  of  their  own  classroom.  Learning  as  a  group  and  sharing 
information  that  could  lead  to  a  stronger  more  responsive  school  was  difficult  for  the 
participants. 

Implications  fpr  the  process  model 

1.  Assessing  readiness  occurs  on  two  levels 

The  facilitator  and  instigators  must  assess  the  availability  of  resources  and 
commitment.  They  will  initially  make  a  "go/no-go"  decision.  Once  the  decision  to  "go"  is 
made  and  the  design  team  formed,  they  must  make  a  different  type  of  assessment,  Thin 
assessment  must  focus  on  the  current  context  and  "what"  should  be  done  and  "how"  should 
they  do  it, 

2.  Preparing  the  participants  and  the  team 
Stabilize  team  membership 

Most  volunteer  groups  will  need  assistance  in  stabilizing  the  team  membership,  A 
few  participants  are  likely  to  drop  out  and  a  few  are  likely  to  join  late,  Closing  any  gaps 
created  by  those  who  leave  and  orienting  those  that  join  late  is  very  important,  Urnup 
productivity  is  improved  by  a  degree  of  cohesiveness.  Participants  coming  and  going  from  a 
group  prevents  cohesiveness  from  developing. 

Develop  knowledge  base 

Community  members  ar.d  teachers  may  not  be  up  to  date  on  what  is  considered 
good  educational  practices.  Helping  them  develop  a  common  understanding  of  instruction,! I 
and  educational  issues  is  crucial  to  their  ability  to  interact  with  stakeholders  during  the 
design  and  implementation  phases. 

Develop  communication  nhills 

A  school  system  may  lack  internal  and  external  communication  skills,  If  I  hey  n  re 
not  in  the  habit  of  communicating  with  parents  and  community  members,  their  skills  will 
likely  be  limited  if  they  exist  at  nil.  In  the  past  some  schools  have  operated  with  very  little 
internal  communication  among  teachers.  Teachers  often  had  little  if  any  time  to 
communicate  with  their  colleagues.  In  fact,  they  often  worked  more  like  isolated  islands 
than  as  colleagues. 

Initially  communication  is  often  viewed  as  a  one-way  process,  "We  will  tell  them 
what  they  need  to  know".  The  concept  of  two-way  communication  is  more  difficult  to  apply. 
The  time  and  energy  required  for  two-way  communication  to  work  is  scarce  in  most  schools 
and  it  is  not  viewed  as  important  enough  to  warrant  the  expenditure  of  such  precious 
resources  as  time  and  energy. 

Communication  appears  continue  to  view  through  the  mind  set  of  independent 
systems  and  one-way  communication  is  sufficient.  Once  the  school  is  viewed  as  an 


563 


interdependent  system  the  importance  and  need  for  two* way  communication  increases,  (thin 
links  to  systems  theory) 

3.  Develop  an  understanding  of  the  current  Context 

Every  school  and  community  tuia  a  past,  u  liiittory  that  shapes  how  it  vinwH  the 
world  and  intcrprctH  eventH.  It  is  crucial  that  the  facilitator  nod  the  design  team  develop  a 
rich  understanding  of  the  multiple  aspects  of  the  current  context.  Much  an  current 
instructional  research  highlights  the  importance  of  meeting  the  learner  where  they  are, 
schools  must  bo  met  whore  they  are.  The  idiosyncrasies  of  a  community  and  a  Hchool  must 
he  considered  when  attempting  to  instigate  change.  They  not  only  have  the  potential  to  trip 
up  the  effort  hut  also  to  offer  keys  to  finding  the  leverage  needed  for  success. 

4.  Develop  a  design  culture 

Systemically  designing  a  school  is  very  different  from  planning  and  implementing 
solutions  to  current  problems.  Creating  a  school  that  is  capable  of  continuous  improvement 
requires  a  culture  that  is  very  different  from  what  currently  exist  in  most  schools.  They 
must  learn  to  move  past  asking  "are  we  doing  things  right",  to  asking  "are  we  doing  the 
right  things  right".  Reflection  and  inquiry  are  essential  to  developing  such  a  culture. 
Expectations  and  support  structures  must  he  put  in  place  that  will  allow  the  participants  to 
develop  the  skills  needed  to  maintain  this  type  of  culture. 


Guidelines: 

A        The  design  team  should  explore  what  it  takes  to  systemically  design  a  school 
system.  Books  and  videos  have  been  developed  that  discuss  continuous  improvement  or 
learning  organizations  and  these  can  he  used  to  generate  discussion  among  the  team 
members. 

B        The  leadership  and  administration  must  exhibit  the  behaviors  that  they  desire  to 
develop  in  the  faculty  and  staff.  Simply  verbally  supporting  two-way  communication  will  not 
he  sufficient.  They  must  seek  and  act  on  input  they  receive  from  all  stakeholders,  including 
teachers  and  parents.  They  should  engage  in  reflection  and  inquiry  and  explain  the 
processes  as  they  use  them.  As  they  carry  out  task  other  members  will  be  performing  in  the 
future,  they  must  take  the  time  to  turn  them  in  to  learning  opportunities.  Teaching  how 
they  performed  the  task  must  become  as  important  as  performing  the  task  itself. 
C         Design  teams  should  be  instructed  in  inquiry  skills.  If  they  are  to  own  and  take 
responsibility  for  the  new  system  they  must  he  allowed  to  collect  and  process  the  data 
themselves.  They  should  be  allowed  to  determine  what  data  they  need  to  collect  and  be 
given  assistance  in  e:  iloring  collection  and  analyzes  methods.  Since  the  schools  will  be  the 
ones  utilizing  the  data  they  must  own  the  collection  process,  the  analysis,  and  the  synthesis 
of  the  data. 

Reigeluth's  Process  Model  with  proposed  mod ifieat ions 
Initiation/Preparation  Phase 

1 .  Assessing  the  readiness  of  the  community, 

*  Facilitator  go/no-go  assessment 

*  Design  team  what/how  assessment 

2.  Get  an  outside  facilitator. 

3.  Get  commitment  from  all  stakeholder  groups. 

*  Get  quality  and  proper  quantity  of  time 
*.        Develop  rich  understanding  of  current  context 
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4 .  Select  an  approach  for  the  change  effort. 

5.  Select  participants  for  the  coordinating  council  and  design  team. 

*Stabilize  team  membership 

6.  Prepare  the  participants. 

*  Develop  knowledge  bases 

educational  and  instructional  methods 
communication 

7.  Relate  with  non  participants. 
*.         Develop  design  culture 

*  Inquiry  and  reflection  skills 

*  Support  structures 
Design/Development  Phase 

S.         Find  common  values  and  analyze  learner  and  societal  needs. 

9.  Develop  core  ideas  and  goals. 

10.  Develop  an  image,  and  design  a  systems  of  functions. 

1 1 .  Design  enabling  systems. 

1 2.  Analyze  the  feasibility. 
Implementation/Documentation  Phase 

13.  Plan  the  implementation, 

14.  Implement  the  design. 

1 5.  Document  the  system. 

Summary 

Restructuring  a  school  system  is  a  difficult  time  consuming  process.  The  nature  of  thf1 
task  places  the  school  in  the  position  of  not  know  where  they  are  going  or  how  to  get  there.  A 
process  to  guide  them  as  the  discover  their  desired  out  come  is  very  important.  This  case 
attempted  to  assistant  an  elementary  school  in  initiating  a  process  and  to  learn  from  their 
experiences  as  a  means  of  helping  other  schools  in  the  future. 

The  additions  to  the  process  model  that  have  been  suggest  are  grounded  in  the 
experiences  of  one  school.  Only  by  using  this  model  with  additional  schools  will  we  learn  if 
they  are  beneficial  and  should  remain  a  part  of  the  model. 

Many  of  the  experience  that  emerged  in  this  case  represent  global  issues  that  may 
have  solutions  in  the  research  of  business,  group  dynamics  and  self-managed  work  groups. 
Drawing  on  the  research  from  other  areas  is  very  important  to  developing  additional 
guidelines  and  refinements.  The  researchers  strongly  encourage  continued  research  in  this 
area  and  the  exploration  of  other  research  findings  in  order  to  better  serve  our  schools  and 
communities  as  they  undertook  the  crucial  task  of  restructuring  education. 
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Introduction 


Purpose  of  the  Study 

The  researchers  examined  an  intensive,  week-long,  adult  training  course  which  was 
taught  with  an  integrated  hypermedia  curriculum.  Trainees'  and  trainers'  use  of  the 
hypermedia  training  materials,  as  well  as  interactions  among  participants  (on-task  and  off- 
task,  during  training  sessions,  and  during  evening  social  activities)  were  recorded  and 
examined. 

There  were  two  main  goals  of  this  research:  To  document  in  detail  how  the 
hypermedia  course  was  implemented,  and  to  formulate  questions  and  hypotheses  about 
aspects  of  learning  with  hypermedia  which  may  guide  future  researchers. 

Outcomes  of  the  Study 
A  comprehensive  research  report  now  being  written  will  describe  the  interactions 
which  took  place  during  the  training  course  in  order  to  convey  the  experience  of  a  participant 
in  this  hypermedia  training  environment.  Cates  (1985)  stated  that  descriptive  studies  of  this 
type  supply  information  which  "contributes  to  the  theoretical  understanding  of  the  ways  in 
which  individuals,  groups,  and  situations  operate"  (p.  95),  This  paper  identifies  some 
questions  and  proposes  hypotheses  about  teaching  and  learning  with  hypermedia  which  may 
be  of  interest  for  closer  examination  in  future  studies.  Due  to  the  constraints  of  this  forum,  it 
summarizes  only  briefly  the  interactions  which  took  place  during  the  training  course. 

Aspects  of  The  Learning  Environment 

Overview 

The  hypermedia  training  environment  evaluated  was  that  of  Cities  in  Schools  (CIS) 
Program  Operations  training,  This  is  a  management  training  course  for  Executive  Directors 
of  local  city- wide  or  county-wide  CIS  dropout  prevention  programs.  The  training  is  carried 
out  at  the  National  Center  for  Partnership  Development  (NCPD)  at  Lehigh  University's 
College  of  Education  in  Bethlehem,  Pennsylvania.  It  is  managed  and  taught  by  CIS  trainers 
utilizing  an  integrated  hypermedia  curriculum  which  was  developed  at  Lehigh's  Advanced 
Information  Technologies  Laboratory.  The  course  is  typically  offered  every  other  month 
throughout  the  year,  and  is  one  of  a  number  of  courses  offered  by  CIS  at  the  NCPD,  and  at 
CIS  s  five  regional  offices. 

Cities  in  Schools 

Cities  in  Schools  is  the  nation's  most  comprehensive  dropout  prevention  program. 
Locally-funded,  not-for-profit  CIS  programs  address  the  varied  needs  of  at-risk  youth  and 
their  families  by  coordinating  existing  human  services  resources,  and  by  repositioning 
service  providers  into  schools  to  work  alongside  teachers  and  other  school  staff.  Cities  in 
Schools,  Inc.,  through  its  national  and  regional  offices  and  the  NCPD,  provides  training  and 
technical  assistance  to  help  local  groups  of  concerned  people  to  replicate  and  operate  CIS 
programs  in  their  communities. 

Advanced  Information  Technologic--  Lahoratorv(AlTL).  Faculty  and  staff  of  Lehigh's 
AITL  have  developed  an  extensive  integrated  hypermedia  curriculum  for  CIS  (Harvey,  1993; 
Harvey  &  Story,  1 !)?)()).  The  hypermedia  developed  for  Cities  in  Schools  consists  of  cross- 
linked  computer,  video,  and  text-based  materials  us  well  as  suggested  implementation  and 
evaluation  strategies  for  trainers.  The  computer-based  textual  and  graphical  resources  refer 
users  to  related  pages  in  print  manuals,  ec  ltrol  a  videodisc  player,  and  prescribe  specific 
learning  activities.  Trainers  may  follow  the  suggested  implementation  plans,  which  are 
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referred  to  as  learning  strategies,  or  they  can  formulate  their  own  strategies  to  suit  their 
needs  or  predilections  (Harvey,  1993,  1993a). 

Program  Operations  Hypermedia  Training 

The  CIS  Program  Operations  course,  a  week-long,  full-day  course  for  CIS  personnel 
who  direct  a  city's  or  county's  CIS  program,  was  the  focus  of  this  study.  The  Macintosh 
computer  software  portion  of  the  CIS  Program  Operations  hypermedia  training  course 
contains  over  2500  different  information  and  activity  screens.  It  accesses  segments  of  full- 
motion  video  and  still  video  slides  from  a  two-sided  videodisc. 

The  print-based  portion  encompasses  over  2000  pages,  in  three  separate  manuals. 

The  Trainee's  Resource  Book  contains  print-based  learning  resources  and  activity 
sheets.  The  Trainer's  Manual  contains  all  the  resources  of  the  Trainee's  Resource  Book,  as 
well  as  the  specific  objectives,  suggested  learning  strategies,  activities,  and  evaluation 
strategies  for  each  of  the  127  learning  sub-activities  which  make  up  the  32  learning 
activities.  The  Monroe  City  Casebook  contains  the  print-based  resources  related  to  a 
simulation  which  is  embedded  in  the  hypermedia  and  integrated  into  many  of  the  learning 
activities. 

The  training  is  organized  into  32  learning  activities.  Each  is  composed  of  three  or 
more  learning  sub -activities  (LSA's);  at  least  one  at  each  of  three  levels  of  learning  specified 
by  Harvey's  (1993)  learning  model: 

Information  acquisition/comprehension,  wherein  trainees  participate  in  activities 

which  help  them  learn  background  and  prerequisite  information  and  skills; 
analysis/synthesis  and  application  in  a  controlled  setting,  which  affords  trainees 

opportunities  to  apply  their  knowledge  and  skills  to  solving  problems  in  a 

simulated  scenario;  and 
action  plan  level,  which  is  the  trainees'  opportunity  to  apply  their  new  knowledge  and 

skills  to  planning  activities  and  improvements  to  be  implemented  on  return  to 

their  home  setting. 

Kach  LSA  in  the  hypermedia  is  designed  as  a  complete  learning  unit.  It  is  focused  on 
the  achievement  of  a  specific  performance  objective.  It  describes  activities  for  trainees  to 
participate  in,  provides  learning  resources  to  use,  and  specifies  performance  criteria  and 
evaluation  strategies  for  trainers  to  use  when  assessing  student  learning  (Harvey,  1993). 

The  training  course.  The  five-day-long  training  course  is  organized  by  CIS  trainers 
who  select  parts  of  the  curriculum  which  they  have  identified  through  a  pre-training 
questionnaire  as  most  requested  for  each  particular  group  of  trainees. 

The  trainers.  Trainers  are  selected  from  Cities  in  Schools,  Inc.'s  staff  of  regional 
trainers  who  normally  work  at  the  five  regional  offices.  They  are  chosen  by  CIS'  training 
coordinator,  based  at  Lehigh  University,  in  consultation  with  CIS'  Vice-President  for 
Training.  There  are  usually  three  or  four  trainers  including  the  training  coordinator,  for  a 
group  of  anywhere  from  ten  to  fifteen  trainees.  Each  trainer  is  assigned  specific  learning 
activities  to  teach  by  the  training  coordinator.  Teaching  style  is  left  up  to  the  individual 
trainer's  predilections  and  judgment.  They  are  free  to  follow  the  learning  strategies 
contained  in  the  hypermedia,  or  not  to. 

The  Setting 

The  National  Center  for  Partnership  Development  (NCPD)  is  a  training  facility 
created  and  maintained  by  a  partnership  of  Cities  in  Schools,  Inc.,  Lehigh  University's 
College  of  Education,  and  Lehigh's  Iacocca  Institute.  The  objective  of  the  NCPD  is  to  utilize 
advanced  technologies  to  assist  learning  and  to  spread  the  knowledge  and  skills  needed  by 
people  from  a  variety  of  sources  and  organizations,  and  by  all  members  of  the  Cities  in 


Schools  fe.nily  in  particular.  The  NCPD  facility  is  located  at  Lehigh  University's 
Mountaintop  Campus  in  Bethlehem,  Pennsylvania.  The  facility  is  composed  of  a  large 
training  room  containing  twelve  Apple  Macintosh  Ilci  computer  workstations  with  videodisc 
capability.  The  workstations  are  located  around  the  periphery  of  the  room.  A  trainer's 
station  is  situated  at  one  end  of  the  room  with  the  capability  to  project  its  computer  display 
and  video  for  large  group  viewing,  and  a  central,  semi-circular  table  is  provided  for  group 
discussions  and  activities. 

Subjects 

The  trainees.  People  who  attend  Program  Operations  training  at  Lehigh's  NCPD  are 
social  service  or  school  administrators,  Executive  Directors  or  board  members  of  local  Cities 
in  Schools  programs,  and  occasionally  people  from  organizations  other  than  Cities  in  Schools. 
Most  have  been  hired  by  a  local  CIS  program's  board  of  directors  to  organize,  set  up,  or 
manage  the  community's  CIS  program.  In  the  past,  trainees  have  come  to  Lehigh  for 
Program  Operations  training  from  all  areas  of  the  United  States  and  from  England,  Canada, 
and  South  Africa.  They  usually  include  members  of  several  cultures,  races,  and 
backgrounds,  and  most  have  attended  college.  All  are  people  who  want  to  help  children  to 
succeed.  Sumo  have  worked  for  social  services  agencies,  and  some  have  worked  for  local 
Cities  in  Schools  programs  in  various  capacities. 

The  training  group.  Each  group  often  to  fifteen  CIS  Program  Operations  trainees  is 
rompo  -I'd  differently  and  has  different  interactions.  The  group  of  trainees  who  participated 
in  the  training  course  examined  was  made  up  of  one  or  two  people  from  each  of  several  of 
different  states,  and  a  small  group  from  a  single  state,  most  of  whom  were  unacquainted. 

Method 
Overview 

The  method  applied  in  this  study  was  developed  for  the  purpose  of  gaining  an 
accurate  image  of  the  occurrences  which  take  place  during  a  Cities  in  Schools  Program 
Operations  training  course.  The  objective  was  to  construct  a  methodology  which  would  allow 
the  researchers  to  gain  as  full  an  understanding  of  the  experiences  of  a  participant  as 
possible,  while  changing  the  environment  as  little  as  possible.  A  major  constraint  on  the 
prospective  methodology  was  that  the  researchers  could  not  interfere  with  or  change  the 
training  program.  The  researchers  determined  that  the  goals  of  the  research  and  the 
constraints  imposed  by  the  setting  could  best  be  accommodated  by  the  application  of  a 
qualitative,  observational  data  collection  method.  Preliminary  data  collected  in  two  pilot 
studies  validated  and  refined  the  method  which  was  to  be  applied  in  this  study. 

Non- Participant  Observation 

The  data  collection  method  consisted  of  the  systematic  observation  of  a  week-long 
training  course  using  a  method  which  is  applied  by  many  researchers  in  the  social  sciences 
(Delamont,  1992;  Miles  &  Hubonnnn,  1984).  Observations  were  recorded  in  brief 
descriptions  and  were  later  expanded  at  the  earliest  possible  time.  When  some  volume  of 
data  had  been  collected,  the  data  were  examined  in  an  attempt  to  identify  explanations  for 
unusual  or  unfamiliar  patterns  or  events.  Initial  interpretations  derived  from  this  analysis 
were  then  reexamined  as  other  data  are  collected  to  test  their  accuracy.  Many  of  these  initial 
interpretations  needed  some  refinement,  and  some  proved  to  be  completely  inaccurate, 

Interview.  In  addition  to  observing  passively,  subjects  were  interviewed  on  specific 
topics  at  appropriate  times  that  did  not  interfere  with  or  interrupt  training  activities  (e.g.  at 


dinner,  or  on  a  break).  The  subjects  were  asked  questions  in  order  to  test  the  validity  of 
preliminary  interpretations  he  had  made. 

End  of  course  reaction  forms.  Although  end  of  course  reaction  forms  are  discouraged 
as  an  evaluation  method  by  a  number  of  educational  researchers  (Dixon,  1987;  Jones,  1990), 
they  are  still  widely  used  in  corporate  management  training  programs  and  many  other 
training  programs.  Put  in  the  proper  perspective,  they  may  have  some  value  as  one  aspect  of 
a  larger  evaluation  methodology  (Ban  &  Faerman,  1990;  Clegg,  1987).  In  the  Program 
Operations  course,  the  extent  of  formal  course  evaluation  being  done  by  CIS's  trainers  was 
collection  of  these  end  of  course  reactions,  and  compilation  of  the  responses.  Since  these  were 
already  a  part  of  the  learning  experience,  the  resultant  data  were  examined,  and  considered 
when  forming  interpretations  and  drawing  conclusions. 

Data  collected.  Categories  of  interactions  \frhich  were  recorded  concerned  the  ways  in 
which  the  hypermedia  course  was  implemented,  and  the  other  activities  which  were 
facilitated  by  the  trainers.  These  categories  were  developed  by  the  application  of  this 
methodology  in  the  pilot  studies  which  were  conducted  during  training  courses  held  in  July, 
1992,  and  in  May,  1993  (Nelson,  1993,  1993a).  The  time  that  each  observation  was  made 
was  recorded,  so  that  the  duration  of  each  interaction  could  be  calculated.  Observations 
made  included  detailed  information  on  the  content  and  context  of  interactions,  including  the 
"ecology1'  of  the  situation  (lighting,  heat,  location  and  movement  of  participants  in  the  room, 
and  other  aspects  of  the  situation  which  may  have  influenced  the  interactions),  and 
concerned  the  categories  listed  in  Table  1. 

Table  1 

Taxonomy  of  Observation  Types 


Individual  activities 

Individual  work  with  computer-based  hypermedia  materials 
Individual  reading  information  from  print-based  hypermedia  materials 
Individual  presents  to  a  small  group 
Individual  presents  to  the  class 
Other  individual  activity 
Small-grow  activities 

Small-group  work  with  computer-based  hypermedia  materials 
Small-group  reading  information  from  print-based  hypermedia  materials 
Small-group  presents  to  the  class 
Other  small-group  activity 

Trainer-led  activities 

Large-group  discussion 

Trainer  providing  help  to  individual  or  small-group 

Trainer  asking  question  of  individual  or  the  group 

Trainer  presenting  object ive(s)  to  the  group 

Trainer  presenting  information  to  the  group 

Trainer  presenting  to  group  with  non-hypermedia  transparency 

Trainer  presenting  to  group  with  flip  chart 

Trainer  directing  trainees  to  read  print-based  hypermedia  resource(s) 
Trainer  presenting  hypermedia  to  group  with  LCD  panel 
Trainer  playing  hypermedia  video  segment  for  the  group 
Trainer  playing  non -hypermedia  video  tape  for  the  group 
Other  observations  and  data 

"Kcology"  of  the  situation  (lighting,  heat,  location  and  movement  of  participants  in 


the  room,  and  any  other  aspects  of  the  situation  which  may  influence  the 
interactions) 
Trainees  taking  a  break 
Trainers'  and  trainees'  comments 
Other  interactions  and  activities 
Responses  to  questions 
Questions  asked 
Trainers'  or  trainees'  answers 
Questions  asked  by  trainees  (and  answers  given) 


Throughout  each  day  of  training,  objective  observations  and  perceptions  were 
recorded.  At  the  end  of  each  day,  the  data  were  classified  into  categories  which  helped  to 
organize  and  group  related  data  to  assist  in  analysis.  An  audio  tape  recorder  was  used  to 
provide  a  more  detailed  record  of  some  group  learning  activities,  and  to  record  comments  and 
suggestions  made  during  "process  checks"  (i.e.  the  last  activity  of  each  day,  during  which 
participants  discuss  the  day's  activities  and  suggest  ways  to  improve  the  training). 

Duration  of  The  Study 

Anthropological  studios  frequently  involve  a  long-term  immersion  in  a  culture  under 
study.  Field  work  lasting  one  to  two  yearn  is  not  unusual.  In  the  field  of  educational 
research,  research  projects  of  this  length  are  impractical.  Educational  researchers  must  deal 
with  problems  such  as  access  to  subjects,  life  cycles  of  innovations  under  study,  and 
insufficient  funding.  Kducationnl  researchers  must  do  more  concise,  focused  studies  (Tucker 
&  Dempsey,  1991),  As  Mac  Donald  (1981)  stated,  "We  go  for  condensed  field  work,  short, 
sharp  bursts  of  data  gathering  leading  to  negotiated  accounts  of  social  action  in  situ"  (p.  4). 

Prior  to  conducting  this  study,  the  authors  examined  two  other  week-long  training 
courses  .  For  the  purposes  of  this  examination  of  training,  the  duration  of  the  data  collection 
phase  comprised  a  five-day  period.  This  was  the  total  time  that  trainees  had  contact  with 
trainers  in  the  hypermedia  course.  The  life  cycle  of  the  course,  and  of  the  training  group 
(trainers  and  trainees),  was  limited  to  these  five  days,  after  which  the  group  disbanded, 
likely  never  to  reconvene  in  the  same  combination.  Although  some  of  the  trainees  had 
attended  other  training  courses,  each  Cities  in  Schools  Program  Operations  training  course  is 
comprised  of  trainees  who  likely  have  never  met  each  other  or  their  trainers,  and  each  course 
is  unique  in  content  and  personalities.  Data  were  collected  throughout  the  training  day,  and 
at  evening  functions,  lunches,  and  breaks. 

Results 
Overview 

The  results  of  this  research  are  currently  being  developed.  The  authors  expect  this 
phase  of  the  study  to  be  completed  by  the  end  of  March,  1994.  Some  preliminary  results  are 
presented  herein,  which  may  guide  others  planning  research  in  this  area. 

Written  Description 

The  constraints  imposed  by  this  forum  restrict  the  researchers  from  providing  a 
complete  written  description  of  the  week- long  Program  Operations  course.  The  final  report 
of  this  study  will  include  a  detailed,  textual  description  and  other  representations  of  the 
events  of  this  week-long  course,  This  paper  includes  n  report  of  the  aspects  of  this  training 
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course  which  appeared  to  warrant  further  investigation.  These  aspects  wore  related  to  a 
number  of  general  areas. 

The  diversity  of  the  trainees  participating  in  the  course  was  extreme,  some  had  held 
their  positions  with  CIS  for  several  years,  others  had  been  hired  only  weeks  or  days  prior  to 
the  course.  Still  others  weren't  even  CIS  employees.  Trainees  came  to  the  course  with  vastly 
different  knowledge,  experiences,  skills,  and  needs. 

This  variety,  and  the  fact  that  some  trainees  worked  in  pairs,  and  others 
individually,  raised  questions  about  the  pairing  or  grouping  of  learners.  It  was  observed  that 
those  who  worked  in  pairs  displayed  a  far  greater  number  of  interpersonal  interactions  than 
people  working  individually,  that  they  frequently  answered  each  other  s  questions  and 
discussed  the  content  of  learning  materials,  and  that  excepting  one  person,  they  participated 
a  great  deal  in  large  group  discussions.  This  aspect  of  learning  with  hypermedia  needs  to  be 
researched  in  more  depth  because,  of  the  thirteen  trainees  in  this  course,  only  four  worked  in 
cooperative  pairs,  and  their  groupings  were  voluntary. 

presented  to  trainees.  Comments  and  questions  which  arose  indicated  that  the 
purpose  of  lessons  was  not  always  made  clear  to  the  learners.  It  is  the  authors'  contention 
that  learners  would  be  more  likely  to  achieve  the  objective(s)  of  any  lesson  if  they  were 
overtly  presented  in  advance. 

Evaluation  of  the  attainment  of  individual  objectives  was  conducted  by  trainers  in 
this  course  without  the  aid  of  formal  tools.  Trainers  analyzed  the  responses  of  trainees 
during  large  group  discussions  and  other  group  activities.  It  was  therefore  impossible  for  the 
trainers  to  assess  objective  attainment  for  each  trainee,  but  only  to  infer  that  if  one  or  more 
trainees  displayed  objective  attainment,  that  all  had  probably  achieved  the  objective(s)  of  the 
learning  activity. 

The  culminating  activity  of  every  suggested  learning  strategy  prescribed  by  the 
hypermedia  curriculum  used  in  this  course  is  the  development  of  an  action  plan  by  each 
trainee.  This  was  described  to  trainees  as  the  writing  of  a  plan  for  implementing  the 
knowledge  and  skills  they  gained  in  the  course.  It  was  to  be  a  detailed  list  of  things  to  do  and 
investigate  upon  return  to  their  CIS  programs.  It  was  observed  however,  that  after  the  first 
day,  this  activity  was  rarely  mentioned,  and  was  skipped  over,  in  spite  of  the  fact  that  it  was 
scheduled  on  the  course  agenda.  On  two  days,  trainees  were  dismissed  early,  despite  the  fact 
that  this  scheduled  activity  was  being  passed  over. 

Although  the  hypermedia  curriculum  used  in  this  course  prescribed  specific  learning 
strategies  for  trainers  to  facilitate,  the  trainers  often  implemented  activities  of  their  own 
design.  When  questioned,  they  reported  that  the  objectives  were  those  listed  in  the 
hypermedia  learning  activities,  but  that  they  had  developed  activities  which  combined  a 
number  of  the  objectives  in  order  to  cover  more  content  in  the  time  available. 

Some  of  the  trainer-developed  activities  involved  the  use  of  learning  resources  which 
trainers  had  developed  in  order  to  supplement  those  available  in  the  hypermedia.  These 
varied  in  content  and  quality. 

After  activities  which  prescribed  trainees  use  of  the  hypermedia  resources,  trainers 
commented  to  each  other  that  they  had  noticed  little  or  no  use  of  print-based  resources  by  the 
trainees.  Although  they  had  observed  this,  and  expressed  that  they  felt  it  was  a  problem, 
they  did  not  instruct  the  trainees  to  more  fully  utilize  these  resources. 

It  has  boon  concluded  that  trainers'  predilections  are  an  extremely  important  factor 
in  determining  resource  utilization  and  learning  of  trainees.  The  instructions  given, 
information  resources  examined,  and  activities  carried  out  are  directly  related  to  each 
trainer's  favored  teaching  style.  Some  trainers  use  hypermedia  ns  an  integrated  part  of  the 
learning  activities  they  facilitate.  Others  use  it  as  an  information  database,  to  provide  rapid 
access  to  information  for  display  to  a  group.  Trainers  who  teach  in  this  way  use  hypermedia 
much  like  an  electronic  flip  chart  or  overhead  transparency.  When  they  allow  trainees  to 


access  hypermedia,  it  is  frequently  in  a  manner  similar  to  the  way  they  might  have  used 
printed  trainee  manuals  in  the  past.  Still  others  avoid  its  use,  preferring  more  traditional, 
trainer  presentation  teaching  strategies.  In  the  course  studied,  each  trainer  utilized 
hypermedia  in  different  ways.  They  varied  their  teaching  and  presentation  styles,  and  did 
not  seem  to  display  a  clear  preference  for  one  particular  mode  of  teaching. 

Time  management  appeared  to  be  difficult  during  this  course.  Several  times 
throughout  the  week,  the  course  agenda  was  revised  to  accommodate  activities  that  took 
longer  or  shorter  than  predicted.  Thursday's  agenda  was  changed  to  reflect  the  trainees' 
needs  with  regard  to  completing  an  out-of-class  assignment,  then  trainees  were  released  from 
class  early,  despite  the  fact  that  they  had  not  worked  on  action  plan  development,  which  was 
on  the  agenda.  Management  of  the  duration  of  breaks  also  seemed  to  be  difficult  in  this 
course.  Many  times,  when  trainees  were  allotted  a  specific  amount  of  time  for  a  break,  they 
didn't  reconvene  until  the  trainers  called  them  back,  often  long  after  the  allotted  time  had 
expired. 

At  the  end  of  each  day  of  this  course,  the  trainers  conducted  what  they  called  a 
"process  check."  This  was  their  way  of  evaluating  that  day's  activities,  and  allowing  trainees 
to  comment  and  offer  suggestions  for  improving  the  activities.  It  seemed  that  this  would 
have  been  an  ideal  time  for  trainees  to  rate  the  relevant  activities  using  their  reaction  forms, 
as  they  had  been  instructed  to  do  on  the  first  day. 

On  Monday,  the  trainees  had  been  told  to  rate  each  day's  activities  on  the  same 
night,  so  that  their  thoughts  and  reactions  would  not  be  forgotten.  Trainers  did  not  remind 
them  to  do  so  however,  so  the  trainees  completed  the  Likert-type  reaction  forms  when  the 
course  was  finished. 

Questions  and  Hypotheses 

Developing  questions  and  hypotheses  about  teaching  and  learning  with  hypermedia 
was  the  major  projected  outcome  of  this  study.  Because  the  use  of  hypermedia  as  an 
instructional  tool  is  so  new,  little  research  has  been  done  regarding  the  ways  that  it  can  best 
be  employed  for  this  purpose.  Some  of  the  unique  attributes  of  hypermedia  afford  users  new 
ways  to  teach  and  to  learn.  It  may  be  that  old  paradigms  of  teaching  and  of  learning  will 
need  to  be  changed  to  most  efficaciously  utilize  hypermedia. 

The  questions  and  hypotheses  which  follow  have  been  derived  by  examining  the 
occurrences  of  a  Cities  in  Schools  Program  Operations  training  course  held  November  8 
through  12,  1993,  and  by  comparing  that  data  set  to  the  occurrences  during  two  pilot  studies 
(Nelson,  1993,  1993a).  They  are  related  to  the  observations  presented  above,  and  concern  a 
number  of  aspects  of  the  use  of  hypermedia  as  a  learning  tool. 

Diversity  of  Trainees 

Hypothesis  1:  There  will  be  no  significant  difference  in  achievement  based  on  trainees*  pre- 
existing levels  of  experience  or  knowledge  in  hypermedia  training  courses  which  include 
people  possessing  various  levels  of  experience  or  knowledge  of  the  subject  matter  . 

Pairing  Learners 

Hypothesis  2:  There  will  be  no  significant  difference  in  achievement  of  course  objectives, 
learning  efficiency,  retention  of  course  content,  or  post-training  job  performance  for  trainees 
learning  with  hypermedia  in  cooperative  pairs  or  those  working  individually. 
Hypothesis  3:  There  will  bo  no  significant  difference  in  the  number  of  requests  for  trainer 
assistance  related  to  course  content  or  to  the  use  of  hypermedia  from  trainees  learning  with 
hypermedia  in  cooperative  pairs  or  those  working  individually. 
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Hypothesis  4;  There  will  be  no  difference  in  the  action  plans  developed  by  trainees  learning 
with  hypermedia  in  pairs,  and  those  of  trainees  who  work  individually. 


Introduction  of  the  Participants 

Question  1:  What  effects  on  the  outcomes  or  objective  attainment  of  trainees  working  with 
hypermedia  do  trainer's  instruction  have? 

Objectives  and  Evaluation 

Question  2:  Are  objectives  for  hypermedia  courses  that  are  developed  by  practicing 
professionals  valid? 

Question  3:  What  is  the  ef  fect  on  post-training  job  performance  of  teaching  practical 
information-retrieval  skills,  rather  than  teaching  specific  information? 
Hypothesis  5:  Trainers  will  more  consistently  implement  learning  activities  designed  to 
expose  trainees  to  content  in  a  shorter  time  than  those  that  take  longer  to  implement, 
regardless  of  the  potential  learning  outcomes. 

Hypothesis  6:  Trainees  will  demonstrate  no  significant  difference  in  objective  achievement, 
retention  of  content,  or  training-related  job  performance  if  the  objectives  of  a  hypermedia 
learning  activity  are  presented  to  them,  or  not. 

Hypothesis  7:  Trainers  will  evaluate  the  objective  attainment  of  trainees  in  a  hypermedia- 
assisted  course  if  they  are  provided  with  easily -applied  measures. 

Hypothesis  8:  When  used  as  an  information  resource,  an  indexed  hypermedia  structure 
facilitates  more  efficient  information  retrieval  than  does  a  hierarchical,  activity-oriented 
structure. 

Hypothesis  9:  There  is  no  correlation  between  one  or  more  trainees  demonstrating 
achievement  of  the  object ive(s)  of  a  hypermedia  learning  activity  in  a  group  discussion,  and 
objective  attainment  by  other  trainees  in  the  group. 

Course  Evaluation  Forms 

Question  4:  If  evaluation  forms  were  completed  after  each  hypermedia  learning  activity, 
rather  than  at  the  end  of  a  course,  would  the  ratings  or  comments  be  different? 
Question  5:  What  is  the  correlation  between  the  trainees'  ratings  of  a  hypermedia  activity, 
and  objective  attainment,  content  retention,  or  successful  post-training  application? 
Question  6:  When  designing  or  modifying  hypermedia  learning  activities,  do  trainers 
formulate  a  lesson  plan  which  includes  one  or  more  objectives  and  evaluation  strategies? 
Hypothesis  10:  A  course  evaluation  which  included  the  objectives  which  were  to  be  achieved 
would  yield  significantly  different  responses  than  a  simple,  Likert-type  reaction  form. 

Action  Plans 

Question  7:  Is  the  development  of  an  action  plan  in  a  hypermedia-assisted  course  correlated 
to  increased  retention  or  application  of  learned  skills  and/or  information? 
Question  8:  Why  was  little  emphasis  placed  on  the  development  of  a  plan  for  implementing 
the  skills  and  knowledge  gained  in  this  hypermedia-assisted  course,  even  though  trainers 
had  earlier  stressed  the  importance  of  such  a  plan? 

Question  9:  Why  are  trainers  reluctant  to  point  out  problems  they  have  noticed  in  each 
other's  hypermedia  learning  activities  or  in  trainees'  performance? 

Trainer-Developed  Activities 

Question  1;  What  effects  on  the  learning  outcomes  or  objective  attainment  of  trainees  do 
trainer's  instructions  have? 

Question  10:  Do  activities  which  are  less-than-directly  related  to  achieving  the  objectives  of 
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training,  but  which  trainees  enjoy,  effective  or  valuable  parts  of  a  hypermedia  training 
course? 

Hypothesis  11:  There  will  be  no  significant  difference  in  trainees'  objective  achievement  from 
hypermedia  learning  activities  whether  they  are  assisted  in  locating  relevant  information,  or 
not. 

Hypothesis  12:  Trainees  are  no  more  likely  to  achieve  the  objectives  of  a  hypermedia 
learning  activity  when  objectives  are  overtly  stated,  than  when  they  are  not. 
Hypothesis  13:  Trainers  will  cover  less  content,  albeit  more  thoroughly,  in  a  hypermedia 
course  which  is  to  be  evaluated  based  upon  trainees*  objective  attainment,  than  in  one  which 
utilizes  "smile  sheets"  (i.e.  Likert-type  reaction,  forms)  for  evaluation. 
Hypothesis  14:  Trainers  who  are  thoroughly  familiar  with  the  resources  available  in  a 
hypermedia  curriculum  are  no  more  likely  to  integrate  their  use  into  learning  activities,  than 
trainers  who  are  not. 

Supplemental  Learning  Materials 

Hypothesis  15:  There  will  be  no  significant  difference  in  learner  achievement  between  a 
course  which  is  taught  with  the  aid  of  projected  hypermedia  resources  and  a  comparable 
course  taught  with  hand-drawn  transparencies. 

Hypothesis  16:  In  a  course  which  is  chiefly  taught  using  computer-generated  teaching  aids, 
the  incongruity  of  hand-drawn  teaching  aids  distracts  learners,  resulting  in  inferior  learning 
of  the  content  of  those  resources. 

Entertaining  Activities  and  Video  Tapes 

Hypothesis  17:  There  will  be  no  significant  difference  in  objective  achievement  by  trainees  in 
a  hypermedia  course  which  follows  lesson  plans  to  the  letter,  or  in_  a  course  which  includes 
entertaining  asides  and  activities. 

Hypothesis  18:  Short,  entertaining  asides  help  trainers  to  gain  and  hold  trainees'  attention, 
thereby  leading  to  more  effective  learning. 

poor  Use  of  Print  Resources 

Question  11:  Do  trainees  prefer  to  use  computer  and  video-based  hypermedia  resources 
rather  than  print-based  resources? 

Hypothesis  19:  The  use  of  print-based  hypermedia  resources  in  addition  to  computer  and 
video-based  resources  does  not  significantly  increase  learning,  retention,  or  post-training 
application. 

Hypermedia  and  Trainers'  Predilection? 

Hypothesis  20:  A  single  training  course  in  the  use  of  hypermedia  for  teaching  is  insufficient 
to  change  trainers'  normal  patterns  of  behavior. 

Hypothesis  21:  There  is  no  significant  difference  in  trainees' objective  achievement  between 
those  taught  with  a  hypermedia  curriculum,  and  those  taught  using  more  traditional,  trainer 
presentation  strategies. 

Hypothesis  22:  Objective  achievement  when  learning  with  hypermedia  is  not  significantly 
different  when  the  hypermedia  is  used  in  a  group  setting  which  allows  for  interpersonal 
interaction,  or  in  an  individualized  setting  without  human  interaction. 
Hypothesis  23:  The  provision  of  sample  forms,  plans,  or  other  materials,  which  demonstrate 
how  others  have  applied  the  content  of  a  hypermedia  course,  will  result  in  increased  post- 
training  application  of  skills  and  knowledge. 
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Time  Management 

hypothesis  24:  There  will  be  no  significant  difference  in  information  retention  or  training- 
related  job  performance  among  trainees  participating  in  a  hypermedia  course  which  covers 
more  content,  and  one  which  covers  less  content  in  more  depth. 

Question  12:  When  a  number  of  hypermedia  learning  activities  are  grouped  and  taught 
together,  are  all  the  objectives  achieved? 

Question  13:  Is  there  any  affect  on  trainees'  attitudes  in  a  hypermedia  course,  when  trainers 
allow  breaks  to  run  longer  than  scheduled? 

Process  Checks 

Question  14:  Do  trainers  change  their  lessons  or  hypermedia  learning  activities  in  future 
courses  that  they  facilitate,  based  on  previous  trainees'  comments? 

Discussion 

These  questions  and  hypotheses  have  been  developed  by  examination  of  the  data 
collected  during  the  November  8  -  12,  1993  Program  Operations  course,  and  by  comparison 
with  data  collected  during  two  other  Cities  in  Schools  training  courses.  A  full  description  of 
the  week-long  course,  a  breakdown  of  each  activity  by  time,  and  support  for  the  questions 
and  hypotheses  developed  will  be  included  in  the  final  report  of  this  study  which  the 
researchers  expect  to  complete  by  the  end  of  March,  1994. 
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Background 


Conceptions  of  teaching,  especially  outside  the  profession,  often  maintain  simplistic 
and  narrow  beliefs  that  teaching  is  a  skill  that  can  be  acquired  relatively  easily  by  any 
reasonably  intelligent  individual.  However,  learning  and  development  theories  suggest  that 
a  complex  skill  like  effective  teaching  is  acquired  over  thousands  of  hours  of  practice 
(Norman,  1978);  practice  that  includes  autonomy,  collaboration,  and  time  (Wildman  & 
Niles,  1987a).  Recent  trends  in  teacher  education  have  stressed  reflection  as  an  important 
component  of  this  learning  process  (Schon,  1991;  Wildman  &  Niles,  1987b).  Helping 
beginning  teachers  leurn  to  think  about  and  analyze  their  pedagogical  activities  is  seen  as 
an  important  step  in  developing  effective  teachers.  The  research  project  described  in  this 
paper  elicits  the  help  of  multimedia  computer  technology  and  modified  case  methodologies  in 
an  attempt  to  encourage  and  facilitate  the  processes  of  reflection  by  beginning  teachers. 

The  theoretical  framework  for  this  effort  is  drawn  from  two  areas  of  research:  the  use 
of  case  methods  in  teacher  education  (Shulman,  1986)  and  cognitive  flexibility  theory  (Spiro, 
et  al.,  1988).  Case  methodology  is  a  proven  instructional  strategy  in  many  areas  (notably 
law,  medicine,  and  business)  that  has  become  more  widespread  in  teacher  education  in 
recent  years.  Many  hold  the  belief  that  case-based  instruction  for  teachers  can  help  them  to 
better  reason  about  and  reflect  on  their  practice  (Christensen,  1987;  Schon,  1983).  A 
number  of  projects  developing  and  using  case  materials  have  been  undertaken  (Greenwood 
&  Parkay,  1989;  Kowalski,  Weaver  &  Henson,  1990;  Silverman,  Welty  &  Lyon,  1992),  whilu 
others  are  advocating  the  use  of  student-written  cases  as  a  means  of  encouraging  beginning 
teachers  to  reflect  on  pedagogical  issues  (Kagan  &  Tippins,  1991;  Shulman,  1991). 

Coincidentally  or  not,  there  is  a  parallel  movement  in  cognitive  psychology  that  is 
emphasizing  cases,  episodes,  or  "stories"  as  a  model  of  storage  and  retrieval  in  human 
memory  (Neisser,  1981;  Schank,  1992;  Spiro  et  al.,  1987).  In  particular,  cognitive  flexibility 
theory  (Spiro,  et  al.,  1988)  proposes  that  the  acquisition  of  knowledge  in  domains  that  are 
ill -structured  (where  content  is  complex  and  application  is  irregular  -  teaching  is  certainly 
such  a  domain)  needs  to  be  decontextualized.  That  is,  rich  interconnections  among 
knowledge  components  need  to  be  developed  in  order  for  learners  to  access  knowledge  from 
multiple  sources  in  the  construction  and  adaptation  of  knowledge  to  new  situations.  Spiro 
and  his  colleagues  assert  that  case-based  instruction  is  likely  to  achieve  the  cognitive 
flexibility  necessary  to  acquire  relevant  knowledge  and  successfully  solve  problems  in  ill- 
structured  domains,  especially  when  the  cases  are  not  organized  into  predefined  categories, 
thereby  requiring  the  learners  to  access  the  information  from  many  different  perspectives. 

Cognitive  flexibility  theory  further  HUggentH  thut  an  appropriate  vehicle  lor  presenting 
the  casta,  and  for  helping  learners  to  "criHMcroHH  the1  landscape"  of  cases  while  noting 
similarities  between  cases  that  on  the  surface  muy  Meem  (liwuimilnr,  is  a  computer-based 
hypermedia  environment  (Spiro  &  Jeting,  1990),  Hypermedia  allows  users  to  rapidly  and 
efficiently  access  a  large  amount  of  information  in  u  nonsequential  manner  by  following 
"links"  (cross  references)  between  "nodes"  (the  actual  information  in  the  database).  Further, 
hypermedia  systems  enn  be  developed  to  allow  learner*  to  create  their  own  links*  between 
nodes,  actively  involving  the  learner  in  reflecting  on  how  vurtouB  aspects  of  one  case  may  be 
similar  to  aspects  of  other  rases,  The  result  in  a  My  stem  that  supports  random  access 
instruction  (Spiro  &  Jehng,  1HW0),  where  learners  build  n  complex  knowledge  representation 
of  the  domain  by  experiencing  canon  from  a  variety  of  perspectives  in  order  to  develop  a  more 
complete  understanding  of  the  nuances  in  the  domain. 
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With  this  theoretical  framework  in  mind,  a  computer-based  multimedia  system  that 
features  prespecified  cases  for  analysis,  facilities  for  linking  various  aspects  of  one  case  to 
other  cases,  and  capabilities  for  entering  and  linking  new  cases  by  individual  students  as  a 
product  of  their  classroom  observations  was  developed  and  tested.  The  software  seeks  to 
effectively  support  the  kind  of  knowledge  acquisition  and  reflection  necessary  to  develop 
sufficient  pedagogical  knowledge  in  preservice  teachers.  The  formative  testing  of  the 
software  was  conducted  to  both  describe  the  kinds  of  activities  undertaken  by  the  students 
when  using  the  system  to  reflect  on  classroom  observations,  and  to  explore  various 
strategies  for  integrating  the  system  into  a  teacher  education  ci  rriculum. 

Description  of  the  Software 

The  software  developed  for  the  project,  Chronicles  of  Teaching,  features  a  design  that 
encourages  users  to  read  and  react  to  descriptions  of  "critical  incidents"  or  mini-cases  that 
typically  occur  in  classrooms.  In  addition,  users  can  enter  descriptions  of  incidents  that  they 
have  observed  during  their  field-based  activities.  Besides  entering  descriptions  of  cases  and 
solutions,  users  may  create  hypermedia  links  between  their  case  and  other  cases  or 
theoretical  descriptions  that  are  in  the  database.  In  this  way,  users  must  reflect  on  the 
similarities  or  differences  between  their  case  and  other  cases  in  the  database  (or 
descriptions  of  theories  and  methods  of  teaching),  and  establish  links  so  that  subsequent 
users  may  follow  the  links  while  examining  the  cases  and  theories  in  the  database.  Another 
feature  that  encourages  reflection  is  a  "notes"  facility  that  can  be  used  to  create  and  print 
personal  notes,  or  to  leave  comments  or  suggested  alternative  solutions  for  other  users  to 
view.  The  software  also  includes  extensive  search  and  indexing  features  so  that  these  cases 
and  theoretical  descriptions  can  be  easily  accessed. 

As  shown  in  Figure  1,  the  structure  of  the  Chronicles  of  Teaching  system  centers 
around  two  database  modules,  cases  and  theories,  that  provide  search  and  indexing 
facilities  to  aid  the  user  in  navigation.  The  "Cases"  module  provides  facilities  for  entering 
descriptions  of  critical  incidents  that  the  user  has  observed  in  classrooms  during  field 
observations  (See  Figure  2).  Students  enter  descriptions  of  both  the  incident  and  the 
solution  to  the  problem,  and  may  also  leave  comments  or  suggested  alternatives  to  cases 
entered  by  others.  A  user  can  also  view  cases  entered  by  other  users  by  following 
hypermedia  links  in  the  database,  searching  for  keywords,  or  selecting  from  an  index,  as 
described  below.  Descriptions  of  the  learning  and/or  instructional  theories  underlying  the 
events  depicted  in  the  cases  are  also  available.  These  descriptions  were  drawn  from  the 
research  literature  on  learning  and  effective  teaching,  including  behavioral  and  cognitive 
theories  of  learning,  motivation  strategies,  questioning  techniques,  classroom  management, 
and  assessment  strategies.  Users  are  encouraged  to  engage  in  debates/analyses  of  the 
cases  by  accessing  the  notes  facilities  or  by  posing  questions  or  leaving  comments  for  other 
users.  Notes  can  be  printed  for  later  examination  and  study  purposes.  Multimedia 
capabilities  to  display  video  clips  of  actual  teac'iing  episodes  are  also  available  to  users. 
These  video  segments  illustrate  various  features  of  some  of  the  cases  and  theoretical 
descriptions  in  the  database.  Facilities  to  allow  users  to  add  video  clips  of  student  teaching 
activities  are  currently  under  development. 
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Figure  1. 

Structure  of  the  Chronicles  of  Teaching  software. 
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Figure  2. 

A  case  from  the  database. 
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Tht?  search  facilities  provided  in  the  software  are  designed  to  allow  easy  access  to  both 
cases  and  theoretical  descriptions.  Searching  for  particular  cases  can  be  accomplished  in 
several  ways.  First,  the  user  can  access  a  search  facility  that  filters  the  available  cases 
based  on  category  selections  made  by  the  user  (See  Figure  3).  The  filters  include  grade  level, 
subject,  and  type  of  teaching  activity.  By  selecting  various  combinations  of  these 
parameters,  the  system  searches  the  database  of  cases  and  displays  the  titles  of  cases  that 
match  the  selected  parameters.  Users  may  then  view  a  particular  case  by  selecting  its  title 
from  the  list.  The  hypermedia  links  embedded  in  the  text  of  the  ease  descriptions  and 
solutions,  as  described  below,  also  provide  an  important  facility  for  accessing  cases.  Each 
ease  in  the  database  also  includes  keywords  describing  various  characteristics  of  the  case 
that  are  entered  by  the  author  of  the  case.  A  uuer  may  access  a  search  facility  that  will 
sequentially  display  all  cases  where  a  match  to  the  current  keyword  is  found.  At  any  time, 
users  may  view  descriptions  of  various  theoretical  principles  drawn  from  the  literature  on 
((Miming,  instruction,  and  effective  teaching.  The  user  simply  selects  the  desired  theoretical 
description  from  a  list  of  descriptive  titles. 
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Figure  3. 


The  cum?  indt»i  screen. 
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The  software  also  provides  capabilities  for  case  authors  to  create  hypertext  links  from 
various  words  or  phrases  in  their  case  to  other  similar  cases  or  theoretical  descriptions, 
thereby  helping  them  to  actively  reflect  on  how  the  events  they  have  observed  in  classrooms 
are  similar  to,  or  different  from,  other  situations  described  in  the  database.  Users  can 
easily  create  links  by  highlighting  a  word  or  phrase  in  the  case  narrative,  navigating  to  the 
target  case,  and  making  a  selection  from  the  menubar  to  create  the  link.  Links  can  also  be 
deleted  by  the  author  of  the  case,  but  once  the  author  is  satisfied  with  the  case  and  links, 
the  text  is  "locked"  so  that  other  users  cannot  alter  the  author's  case  description  or  links. 

Field  Testing 

Development  of  the  Chronicles  of  Teaching  software  has  proceeded  using  a  rapid 
prototyping  model.  That  is,  more  than  45  teacher  education  students  from  the  target 
population  have  participates  in  several  cycles  of  testing  and  revision  of  the  software. 
Initially,  randomly  selected  students  from  sections  of  the  teacher  education  program  who 
had  completed  16  hourw  of  course  work  in  education  (Introduction  to  Education,  Educational 
Psychology,  etc.),  and  who  were  entering  a  "block*  of  methods  courses  in  elementary 
education  that  requires  extensive  observation  in  public  school  classrooms  (20  hours  per 
week),  helped  with  testing  of  the  various  prototypes.  Participants  were  introduced  to  the 
system,  and  to  case  writing  and  analysis  procedures,  during  a  training  session  prior  to  their 
scheduled  observations  in  the  schools.  The  students  then  used  the  system  individually  to 
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enter  mul  link  cases  and  solutions  based  on  their  classroom  observations  during  the 
remainder  of  the  period  of  classroom  observations. 

The  major  activities  completed  by  each  student  involved  composing  and  entering  their 
own  ease(s)  derived  from  their  classroom  observations,  and  linking  various  aspects  of  their 
case(s)  to  other  cases  and  to  the  theoretical  descriptions  included  in  the  system  so  that 
subsequent  users  could  access  the  student's  case(s)  from  a  variety  of  other  poirr  j  in  the 
hypermedia  database.  The  students  also  responded  to  other  cases  by  viewing  and 
commenting  on  analyses  made  by  other  users.  The  system  was  instrumented  to  collect  data 
regarding  the  specific  activities  each  user  completed  during  interaction  with  the  system. 

Feedback  from  the  students  who  tested  the  system  was  used  to  guide  modifications 
and  enhancements  of  the  prototype.  Specifically,  users  asKed  for  a  more  sophisticated 
search  system  that  would  allow  them  to  search  for  cases  based  on  various  categories  of 
teacher  activities  (i.  e.,  discipline,  instruction,  classroom  management,  etc.)  as  well  as 
student  level  (i.  e.,  early  elementary,  middle  school,  high  school,  etc.)  and  context  (i.  e.,  math, 
reading,  science,  recess,  etc.).  This  search  feature  was  added  to  the  system,  along  with 
navigation  capabilities  that  allow  users  to  "backtrack"  along  the  path  they  had  followed 
while  browsing  with  hypertext  links. 

Besides  specific  suggestions  regarding  modifications  of  the  system,  students  also 
volunteered  their  subjective  impressions,  noting  that  the  tasks  involved  with  using  the 
system  were  very  appropriate  for  reflecting  on  various  aspects  of  teaching.  The  activities 
involved  in  entering,  linking,  analyzing  and  commenting  on  cases  in  the  database  seemed  to 
engage  the  students  at  n  high  level.  One  student  even  jokingly  suggested  that  field 
observations  be  replaced  with  use  of  the  system  because  the  same  outcomes  could  be 
achieved  with  a  significant  reduction  in  mileage  on  his  cart 

 A  more  extensive  analysis  of  the  cases  entered  during  testing  of  the  prototype  reveals 

that  the  students  arc*  noticing  and  discussing  a  wide  variety  of  classroom  situations  in  both 
early  and  late  primary  classrooms.  An  examination  of  the  topics  for  the  32  cases  entered  in 
the  database  reveals  that  discipline  (47%)  and  instructional  methods  (31%)  dominate  the 
concerns  of  the  preaorvico  teachers  who  have  tested  the  prototype,  while  the  context  of  the 
incidents  is  more  diverse,  covering  reading,  math,  science,  history,  language  arts, 
health/physical  education,  and  even  recess.  It  is  also  apparent  that  the  system  encourages 
the  kind  of  reflective  thinking  necessary  to  develop  teaching  expertise,  as  exemplified  by  the 
following  excerpt  from  an  opinion  entered  by  someone  in  response  to  a  ease  description: 

I  think  proximity  would  be  the  best  in  this  situation.  By  moving  closer  to  the 
students  and  standing  next  to  them,  they  will  know  that  they  were  heard.  Some 
teachers  may  separate  them,  but  this  causes  disturbance  to  the  lesson  flow.  I 
would  continue  the  lesson  while  moving  towards  the  student. 

Integration  into  the  Teacher  Education  Curriculum 

The  Chronicles  of  Teaching  software  has  been  integrated  in  several  courses  of  the 
Momentary  Education  Program  at  Southern  Illinois  University  at  Edwardsville,  thereby 
involving  all  elementary  education  majors  in  four  stages  of  utilization  of  the  software.  In  the 
first  stage,  students  enrolled  in  the  Foundations  of  American  Education  course  (the  initial 
course  for  all  education  majors)  are  introduced  to  the  software  through  discussions  of  the 
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cases  already  in  the  databases.  The  software  is  demonstrated  by  the  instructor,  and 
students  are  then  encouraged  to  browse  through  the  databases  and  enter  opinions  and 
alternative  ideas  for  solutions  of  the  problems  described  in  the  cases.  In  this  way,  the 
students  are  encouraged  to  reflect  on  the  situations,  and  to  compare  what  is  described  in  the 
cases  to  what  they  see  during  the  20  hours  of  field  observations  that  are  required  in  the 
course. 

After  the  introductory  course,  elementary  education  majors  enroll  in  a  general 
methodology  course  where  they  are  required  to  write  a  description  of  a  critical  incident 
("mini -case")  they  observed  during  their  field  work  in  the  public  schools.  The  students  enter 
their  case  description  into  the  database  and  continue  to  examine  the  other  incidents  that 
are  already  entered  in  the  database.  Part  of  the  process  involved  in  creating  a  new  case 
requires  the  students  to  link  aspects  of  their  description  to  other  cases  in  the  database, 
supporting  a  reflective  process  whereby  they  identify  similarities  and  differences  between 
what  they  observed  and  what  others  have  observed.  Particularly  important  in  this  reflective 
process  is  the  consideration  of  how  the  situation  was  resolved,  and  the  alternative  solutions 
that  might  be  considered.  Students  also  continue  to  respond  to  other  cases  in  the  database 
by  viewing  and  commenting  on  analyses  made  by  other  participants. 

The  third  stage  of  integration  of  the  Chronicles  of  Teaching  software  into  the  teacher 
education  curriculum  occurs  in  several  of  the  five  methods  courses  that  students  must 
complete.  For  example,  in  the  Social  Studies  Methods  class,  students  are  required  to  write 
two  critical  incidents  they  have  observed  during  their  100  hours  of  classroom  experiences 
completed  during  this  blo^k  of  the  program.  The  critical  incidents  at  this  phase  might 
describe  incidents  they  have  observed  or  that  occurred  while  they  were  actually  teaching. 
Again,  participants  are  encouraged  to  respond  to  other  cases  by  viewing  and  commenting  on 
analyses  of  the  other  cases  in  the  database. 

The  last  stage  of  curriculum  integration  takes  place  during  student  teaching.  All 
elementary  student  teachers  are  required  to  produce  a  description  of  a  critical  incident  that 
occurred  during  their  student  teaching  and  place  it  in  their  final  senior  student  teaching 
portfolio  as  well  as  enter  the  case  in  the  database.  As  time  permits,  these  students  are  also 
encouraged  to  continue  responding  to  other  cases.  Hopefully,  student  teachers  and  even 
alumni  will  continue  to  take  advantage  of  the  case  descriptions  collected  in  the  database  to 
reflect  on  their  teaching  and  learn  from  the  analyses  and  alternatives  suggested  by  users  of 
the  Chronicles  of  Teaching  software. 
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Introduction:  Thuurutinal  lWnpei!Uvn  and  Objectives  of  Study 


This  is  an  analysis  of  experimental  ntitcuineH  that  resulted  In  different  Hynergistic 
performance,  motivational,  ittul  learning  Htrateglen  relatioiiHhips  between  and  among  the 
variables  studied  in  two  groups, 

Tin*  hy|Mit Iii»hIh  nf  (ho  study  wuh  that  subjects  trained  in  seleeted  task-related  learning 
strategies  would  exhibit  significantly  increased  performance  and  academic  motivation  over 
untrained  subjects,  Previous  research  had  recommended  learning  Mtratogy  interventions 
bused  on  reported  mrrclat  iotin  between  selected  learning  strategies,  performance,  and  self- 
concept  related  motivational  dimensions  (See  Eison,  Pollio,  &  Milton,  1986;  McCombs,  1984, 
I1IH7;  Pintrich,  1980,  1987;  Pintrich  &  De  Groot,  1988;  Schutz,  Ridley,  Glanse,  &  Weinstein, 
19H9), 

According  to  a  social-cognitive/expectancy-value  model  of  college  learning  and  teaching 
developed  by  McKeachie,  Pintrich,  Lin,  &  Smith  (1986),  student  motivation  and  knowledge 
of  learning  strategies  not  only  affect  each  other,  but  lead  to  both  self-regulated  learning 
(engagement  in  learning  or  application  of  learning  strategies)  and  academic  performance. 
Individual  student  characteristics  (including  personal  academic  beliefs,  attribution  to 
success  and  failure,  and  demographics),  however,  mayinfluence  task-related  motivation, 
and  academic  performance. 

Figure  1  presents  a  theoretical  framework  of  the  study,  and  Figure  2  diagrams 
experimental  variables. 

The  task  was  a  management  case  study.  As  shown  in  Figure  2,  training  was  provided 
in  the  use  of  techniques  related  to  five  types  of  cognitive,  metacognitive,  and  resource 
management  learning  strategies.  Performance  was  defined  as  scores  on  three  parts  of  a 
case  study  task. 

Motivation  was  defined  as  scores  on  each  of  three  self-report  measures  of  motivation:  (a) 
perceived  task- related  self-efficacy  for  learning,  (b)  expectations  for  task-specific  academic 
success,  and  (c)  desire  to  continue  learning  more  about  business  management. 

Two  categories  of  learning  strategies  were  assessed:  (a)  "primary11  learning  strategies 
(general  cognitive  and  metacognitive  strategies,  classified  as  self-regulated  learning 
atrntegicH  by  Pintrich  et  al.,  1989),  and  (b)  "other"  learning  strategies  used  and  valued 
(specific  study  skills-like  tactics). 

Operational  definitions  for  all  variables  appear  in  Appendix  A. 

Methods  and  Data  Sources 

Three  sections  of  an  Associate  Degree  introductory  management  course  volunteered  to  be 
subjects  (N  =  I    ■  they  were  located  on  two  regional  campuses  of  a  comprehensive  state 
university.  1        ie  students  in  each  class  were  randomly  assigned  to  be  subjects  in  either 
a  control  group  or  a  treatment  group. 

Figure  1 
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Figure  2 


In  a  posttest-only  two-group  experimental  design,  both  groups  received  training  in 
management  case  study  analysis  over  a  two- week  period  involving  four  75-minute  class 
sessions.  In  addition,  the  treatment  group  received  training  in  learning  strategic"*.  All 
training  was  transmitted  via  written  materials  developed  and  validated  by  the  research 
investigator  via  expert  review  and  extensive  pilot  testing  (See  Nuttall,  1993). 

Performance  was  assessed  by  expert  grading  of  open-ended  answers.  A  Urn-pan*1  self- 
report  questionnaire  was  used  to  gather  other  data  (learning  strategy  use,  learning  strategy 
value,  individual  characteristics,  and  motivation  outcomes).  MANOVA  watt  used  to  hhhiwh 
group  differences  at  a  .05  significance  level.  Follow-up  interviews  were  conducted  with  1 7 
percent  of  the  sample. 

Sources  for  the  questionnaire  included:  (a)  adaptations  (from  context  of  courso  to  tusk)  ol 
the  Motivated  Strategies  for  Learning  Questionnaire,  (MSLQ,  Pintrich,  Smith,  &  MrKeneliie, 
10R9),  (b)  the  Achievement  subscale  of  the  MultidimcnBional-Multiattrilnitiohiil  ('utiHitlity 
Scales  (MMCS-IV,  Lefcourt,  Von  Baeyer,  Ware,  &  Cox,  1979),  and  (c)  invoKHKUtor-propmril 
questions.  Doth  intoritem  alpha  and  test-rctest  Pearson  r  reliabilities  wt»h«  compiled  tor  nil 
questionnaire  items. 

Liiniluiuaui 

Limitations  IiicIiicIimI  piiHHllili*  offerls  of  time  ami  iihi1  of  sell  report  n hhi'h h ii i < 1  m t 
I'lOireming  (Imp,  it  wiih  hated  thut  pilot  studios,  which  took  place  over  longer  periods  ol 
Iiiih1,  showed  sianilinint  ^ahis  in  plaunlnk*.  Kolfeffieaey,  ami  export  miry  for  staves*  Ah 
related  to  ueruruto  reporting  hy  subjects,  limitiit Ioiih  wore  considered  as  related  to  sueta! 
desirability  ol'  responses,  accuracy  of  perceptions  of  oho,  mill  semantics. 

Hummarv  of  Ueaults 

 A  Hti imii/ii'v  ol  results  lire  presented  for  performance,  motivation,  learning  strategies  use, 

nod  interactions  ol  variables  (Hoe  also  Nuttall,  1992), 

ri'rformaiuru 

The  treatment  resultoil  in  slKhificnnt  performance  increases  in  two  tasks:  (a)  listing 
symptoms  of  r,.«i»  problemls),  anil  (b)  labelling  symptoms  according  to  management '  action 
or  process.  There  wiih  no  siMnllicunt  increase  for  a  third  task,  writing  a  problem  statement. 

Motivation 

CnncornihK  the  throe  motivation  outcomes,  there  were  no  significant  increases.  However, 
row  scores  on  sell'  efficacy  nod  expectancy  for  success  (which  were  correlated  with  each  other) 
won1  slightly  higher  for  tin*  treatment  group;  and  follow-up  interviews  with  subjects 
indicated  that  they  felt,  tin*  one  of  learning  strategies  was  motivating. 

Furthermore,  analyses  of  interactions  between  all  the  variables  in  the  study  (discussed 
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below)  indicate  that  motivation,  along  with  individual  characteristics,  played  a  key  role  in 
performance  outcomes  among  treatment  group  subjects,  albeit  in  an  indirect  way.  In 
particular,  correlations  between  self-regulated  learning  (primary  learning  strategies)  and 
motivation  were  stronger  for  the  treatment  group. 

Supported  by  results  obtained  in  a  pro-pilot  ntudy,  it  ih  anticipated  that,  with  additional 
time,  motivational  results  would  have;  been  significant,  at  least  in  the  areas  of  self-efficacy 
and  expectancy  for  success. 

Learning  Strategies  Use 

Post  hoc  analyses  indicated:  (a)  Hignifinint  lines  of  two  "other"  learning  strategies 
(referring  frequently  to  inHtructioiiH  and  other  material,  and  reading  the  cse  more  than 
once),  and  (b)  tendencies  toward  more  overall  learning  Htrategy  use  in  the  treatment  group. 

Interactions  of  Variables 

Theoretical  relationship!*  between  the  following  net**  of  variables  were  supported:  (a)  self- 
regulated  learning  and  motivation,  (b)  Keif- regulated  learning  and  individual  characteristics, 
(c)  functions  aspect  of  task  and  selfefficary,  and  (d)  task-related  motivation  and  individual 
characteristics.  There  was  no  support  for  the  relationship  between  self-regulated  learning 
and  performance. 

Synergistic  Interactions  of  Variables  Within  Groups 

In  this  study,  the  learning  strategies  troutment  appears  to  have  caused  differences 
between  the  groups  that  were  not  obvious  from  merely  looking  at  significant  differences  for 
performance  and  motivation  outcomes.  Figures  3  and  4  illustrate  complex,  synergistic 
relationships  at  work  in  the  "black  boxes"  of  the  mind  between  learning  strategy  value, 
individual  characteristics,  motivation,  and  learning  strategy  use.  These  figures  are  entitled 
"General  Relationships..."  because,  for  the  learning  strategies  variables,  only  general 
categories  are  provided  rather  than  specific  strategies. 

Fifteen  areas  where  correlations  were  unique  or  higher  in  the  control  group  are  indicated 
in  Figure  3  as  Letters  D,  F,  G,  I,  L  through  Q,  and  R  through  V.  Seventeen  areas  where 
correlations  were  unique  or  higher  for  the  treatment  group  are  indicated  in  Figure  4  as 
Letters  A  through  Q. 

To  show  relationships  between  variables,  Pearson  correlation  coefficients  were  computed 
by  individual  subject.  Two-tailed  significance  at  the  .05  level  was  computed  for  each  group- 
control  (S)  and  treatment  (T)  as  well  as  for  the  total  sample. 

Although  individual  correlations  are  all  significant  at  the  .05  level,  no  between-group 
significance  statistics  were  calculated.  Differences  were  determined  by  observing  whether: 
(a)  one  group  had  a  correlation  between  two  variables  that  the  other  group  did  not  have  (a 
unique  correlation),  or  (b)  one  group  had  a  noticeably  higher  (around  .1000  higher) 
correlation  than  the  other. 

Of  the  19  correlations  that  are  indicated  as  being  higher  for  one  group  than  the  other,  six 
differences  range  from  .0638  to  .0968,  and  13  differences  range  from  .1126  to  .1710. 
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A  summary  of  unique  and  higher  correlations  between  the  groups  appear  in  Appendix  B, 
Tables  1  through  11.  A  brief  summary  of  types  of  differences  between  the  treatment,  Group 
T  (Figure  3),  and  the  control,  Group  S  (Figure  4),  appears  below  in  alphabetical  order, 

A,  B,  and  C  signify  Group  T's  unique  correlations  between  motivation  outcomes  (self- 
efficacy,  expectancy  for  success,  and  desire  to  continue  learning)  and  self-regulated  (primary) 
learning  strategies  (See  Table  1). 

Figure  3 

Figure  4  ' 

D  signifies  unique  correlations  for  both  groups  between  self-regulated  (primary)  learning 
strategies  and  individual  characteristics  (See  Table  2). 

E,  F,  and  G  signify  correlations  between  motivation  and  individual  characteristics  (See 
Table  3),  Only  Group  T  has  unique  correlations  between  self-efficacy  and  individual 
characteristics  (E),  while  both  groups  have  unique  correlations  between  individual 
characteristics  arid:  (a)  expectancy  for  success  (F),  and  (b)  desire  to  learn  (G). 

i  I  indicates  unique  correlations  between  symptoms  and  other  learning  strategies  valued 
for  ( i roup  T,  while  I  indicates  unique  correlations  for  both  groups  between  problem  and 
individual  characteristics  (See  Table  4). 

•I  and  K  indicate  unique  correlations  for  Group  T  between  desire  to  learn  and:  (a)  self- 
elfirncy  and  (b)  expectancy  for  success,  respectively  (See  Table  1). 

Ij  indicates  unique  correlations  for  both  groups  between  self-efficacy  and  other  learning 
stratifies  use  (See  Table  5). 

M,  N,  and  ()  signify  differences  for  both  groups  in  correlations  between  other  learning 
strategies  valued  and:  (a)  self-efficacy  (M), 

(h)  expectancy  for  success  (N),  and  (c)  desire  to  learn  (0).  (See  Table  6.) 

V  indicates  differences  in  both  groups  between  individual  characteristics  and  other 
learning  strategies  used.  Group  S's  correlations  appear  in  Tables  3  and  4,  while  Group  T's 
correlations  appear  in  Table  9. 

Q  indicates  that  Group  T  exhibited  higher  correlations  between  other  learning  strategy 
use  and  value  (See  footnote  in  Table  5  plus  Tables  10  and  11). 

R  indicates  Group  S's  unique  correlations  between  functions  and  self-efficacy  (See  Table 

4). 

S  indicates  Group  S's  unique  correlations  between  problem  and  other  learning  strategies 
used  (See  Table  4). 

T  indicates  Group  S's  unique  correlations  between  problem  and  other  learning  strategies 
valued  (Sec  Table  4). 


U  and  V  indicate  Group  S's  unique  correlations  between  other  learning  strategies  used 
and:  (a)  expectancy  for  success  and  (b)  desire  to  learn,  respectively  (See  Table  5). 

Analysis  of  Group  Differences:  Comparing  "Black  Boxes" 

After  extensive  post  hoc  analysis  of  the  data,  it  is  evident  that  there  are  complex 
explanations  of  the  outcomes.  As  Dansereau  (1985)  has  obserx  ed: 

We... know  very  little  about  the  relationships  between  individual  difference  variables  and 
training  effectiveness.. .(and)  about  why  the  strategics  work.  We  don't  know  what 
alterations  in  the  participant's  cognitive  processing  are  responsible  for  improved  test 
performance  (pp.  230-231). 

To  interpret  interactions  between  and  among  variables  in  this  study,  and  to  thus 
analyze  what  is  occurring  in  each  group's  "black  box,"  (See  Figures  3  and  4),  the  following 
categories  of  relationships  are  discussed:  (a)  Role  of  other  learning  strategist  valued,  (b) 
other  learning  strategies  used  and  individual  characteristics,  (c)  self- regulated  learning  and 
individual  characteristics,  (d)  motivation  and  individual  characteristics,  h)  association  of 
motivation  outcomes.with  each  other,  (f)  motivation  and  self-regulated  learning,  (g) 
motivation  and  other  learning  strategies  used,  and  (h)  associations  v/ith  performance 
outcomes. 

After  empirical  results  are  discussed,  a  theoretical  discussion  of  the  interaction  of 
variables  within  groups  is  presented  in  the  form  of  a  hypothetical  model.  This  model 
attempts  to  explain  processes  occurring  within  the  "black  boxes." 

Role  of  Other  Learning  Strategies  Valued 

The  role  of  learning  strategy  value  was  evident  in  several  unique  correlations  in  the 
treatment  group.  First,  the  treatment  group  had  higher  correlations  of  value  with  uho  (Hce 
Tables  10  and  11)  for  selected  strategies.  Second,  the  treatment  group  had  a  unique 
correlation  of  symptoms  and  one  learning  strategy  valued  (Value  of  asking  othorH  for  help),  u 
strategy  presented  to  the  treatment  group  and  not  the  control  group  (Table  4,  Code?  II), 

Third,  the  treatment  group  had  more  unique  correlations  of  learning  strategic  valued 
with  self-efficacy  and  expectancy  for  success  (Table  6,  Codes  M  and  N). 

Fourth,  although  both  experimental  groups  exhibited  unique  associations  between 
motivation  outcomes  and  other  learning  strategies  valued,  there  were  differences  in 
particular  strategics  and  specific  motivation  outcomes  correlated  (See  Table  6). 

Evidently  the  treatment  evoked  these  associations,  since  all  the  strategies  associated 
with  all  three  motivational  outcomes  by  the  treatment  group  were  recommended  in  the 
learning  strategies  intervention. 

Other  Learning  Strategies  Used  and  Individual  Characteristics 

Tables  7  and  8  (Code  P)  show  the  control  group's  unique  associations  between  other 
learning  strategies  (12  particular  skills)  used  and  individual  characteristics/demographics, 
while  Table  9  (Code  P)  exhibits  the  treatment  group's  unique  associations. 


The  treatment  group's  correlations  between  other  learning  strategies  used  and  individual 
characteristics/demographics  were  generally  limited  to  task  utility,  task  importance,  and 
overall  task  value  (interest,  importance,  and  utility).  There  were  no  associations  of  other 
learning  strategy  use  with  the  following  individual  characteristics;  task  interest,  test 
anxiety/cognitive  interference,  or  generally  any  type  of  causality  beliefs  such  oh  effort, 
internal  or  external  beliefs,  and  attributions  to  failure  experience. 

On  the  other  hand,  for  purposes  of  discussion,  control  group  correlations  may  be  grouped 
into  four  categories.  In  the  first  category  are  thos<*  characteristics  that  seem  to  indicate  that 
learning  strategies  are  not  generally  helpful  and  are  used  only  as  a  last  resort  to  avoid 
failing:  (a)  overall  test  anxiety,  (b)  low  GPA,  (c)  no  prerequisite  Introduction  to  Business 
course,  (d)  attributions  to  failure  experiences,  and  (e)  external  causality. 

The  second  category  indicates  ease  in,  or  time  available  for,  using  the  strategies:  low 
semester  credit  load.  The  third  category  of  individual  characteristics  urates  to  interest  in 
task:  (a)  task  value  (interest)  and  (b)  negative  interest  in  study.  The  fourth  category 
indicates  particular  types  of  students:  (a)  high  attendance  and  (b)  male. 

Self-Regulated  Learning  and  Individual  Characteristics 

As  indicated  in  Table  2  (Code  D),  the  treatment  group  exhibited  more  than  30  unique 
correlations  between  self-regulated  learning  and  individual  characteristics,  while  the  control 
group  exhibited  only  two  such  correlations. 

While  the  control  group's  associations  of  individual  characteristics  with  primary  learning 
strategies  were  limited  to  effort  and  task  utility,  the  treatment  group  exhibited  more 
combinations  of  personal  academic  beliefs  and  learning  strategy  use  that  indicated 
tendencies  toward  intrinsic  motivation  and  overall  task  value;  and  they  seemed  to  value  the 
whole  learning  experience  more,  including  both  task  completion  and  learning  strategy  use. 

Motivation  and  Individual  Characteristics 

Since  motivation  and  self-regulated  learning  variables  are  correlated  (See  Table  1), 
similar  relationships  betweer  motivation  and  individual  charactertBt-cs  could  he  expected, 
which  was  the  case  in  this  study.  As  indicated  in  Table  3,  for  the  control  gn      there  were 
only  four  unique  correlations  between  motivation  and  individual  chnracteri..'.       On  the 
other  hand,  the  treatrient  group  had  more  numerous  and  complex  unique1  correlation* 
between  motivation  and  individual  characteristics,  as  explained  below. 

Individual  characteristics  associated  with  self -efficacy.  As  shown  under  (%>de  K,  the 
learning  strategies  treatment  seems  to  have  evoked  associations  with  self-efficacy  of  overall 
task  value  (particularly  interest  and  utility)  a*  well  as  a  tendency  to  not.  associate  external 
events  or  luck  with  academic  performance. 

Individual  characteristics  associated  with  expectancy  for  success.  An  shown  under  Code 
F,  the  learning  strategies  treatment  seems  to  have  evoked  associations  with  expectancy  far 
success  that  were  identical  to  self-efficacy  associations  in  all  but  two  aspects:  tank  utility 
and  overall  task  value  (There  was  no  difference  between  the  control  and  treatment  groups 
for  these  two  associations). 
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Individual  characteristics  aaaflCifltcd  with  deaire  to  learn.  As  shown  under  Code  Gf  the 
learning  strategies  treatment  seems  to  have  evoked  more  associations  in  both  groups 
between  desire  to  learn  and  all  individual  characteristics  than  other  motivation  outcomes 
did.  Also,  similar  to  self-efficacy  and  expectancy  for  success  associations,  the  treatment 
seems  to  have  evoked  an  association  with  desire  to  learn  of  not  associating  external  events, 
luck,  or  past  failure  with  academic  performance.  Again,  the  control  group  associated  desire 
to  learn  with  such  variables  as  task  utility,  effort  beliefs,  and  internal  causality  beliefs. 

(Years  since  high  school  is  an  unexplainable  association  with  desire  to  learn  for  both 
total  sample  and  the  treatment  group  unless  it  indicated  the  experimental  groups  were  not 
equivalent  in  some  combination  of  variables,  such  as  gender  and  age.) 

Associations  of  Motivational  Outcomes  with  Each  Other 

 In  the  treatment  group,  there  were  higher  correlations  between:  desire  to  learn  and:  (a) 

self-efficacy  and  (b)  expectancy  for  success  (See  Table  1,  Codes  J  and  K). 

Motivation  and  Self-Regulated  Learning 

As  shown  in  Table  1  (Codes  A  and  B),  the  treatment  group  had  unique  correlations 
between  the  two  motivational  variables  of  self-efficacy  and  expectancy  for  success  and  the 
two  self-regulated  learning  variables  of  planning  and  regulating.  The  treatment  group's 
correlation  between  desire  to  learn  and  all  self-regulated  learning  strategies  were  also  higher 
(Table  1,  Code  C). 

Motivation  and  Other  Learning  Strategies  Used 

As  indicated  in  Table  5  (Code  L),  the  treatment  group  had  only  two  unique  correlations: 
(a)  Self-efficacy  and  writing  notes  in  the  margin  and  (b)  self-efficacy  and  referring  frequently 
to  instructions  and  other  materials.  However,  these  strategies  were  ranked  as  high-use  and 
high-value  by  both  groups,  There  were  no  treatment  correlations  between  expectancy  for 
success  or  desire  to  learn  and  oth*?r  learning  strategies  used. 

In  the  contrjl  group,  two  strategies  were  associated  with  both  Helf-efficacy  and 
expectancy  for  success  (Codes  U  and  V):  Timing  work  and  making  a  diagram  of  cane  facts, 
both  which  were  low-use,  low-value  ranked  strategies  by  both  groups. 

It  appears  that  the  treatment  group  associated  referring  frequently  to  instructions  and 
other  materials  with  personal  ability  to  perform  a  task  (self-efficacy),  whereas  the  control 
group  associated  the  same  learning  strategy  with  personal  interest  in  the  task  or  desire  to 
learn  more  about  it. 

It  appears  that  self-efficacy  may  have  been  the  key  to  increased  performance  in 
symptoms  and  functions,  since  it  was  linked  with  the  only  learning  strategy  used 
significantly  mr  e  by  the  treatment  group.  Furthermore,  self-efficacy  was  linked  to  functions 
for  the  total  sa  nple,  and  functions  and  symptoms  were  linked  for  all  groups. 
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Associations  with  Performance  Outcomes 


The  following  associations  (shown  in  Table  4)  may  have  influenced  better  performance 
outcomes  for  symptoms  and  functions  in  the  treatment  group. 

1.  The  association  between  symptoms  und  valuing  asking  others  for  help  (Code  H).  This 
strategy  appeared  in  the  treatment  instructions  although  it  could  not  be  used  unless 
students  talked  outside  of  clans  (the  experimental  site);  however,  the  rationale  for  its 
leading  to  increased  performance  would  bo  Hupported  by  Dansereau's  (1985)  finding  that 
peer/dyad  learning  leads  to  increased  achievement. 

2.  The  inclination  to  not  associate  self-efficacy  with  performance  (Code  R).  One  reason 
for  this  situation  may  have  been  that,  while  both  groups  received  feedback  on  their 
performance  throughout  the  experimental  training  period,  the  treatment  group  did  not  worry 
about  getting  lo-  scores  because  they  knew  they  had  an  arsenal  of  learning  strategies  that 
would  help  thei  .  perform  better.  In  effect,  the  treatment  group's  self-efficacy  would  appear 
to  be  higher,  despite  non-significantly  higher  raw  scores. 

Theoretical  Discussion  of  Interaction  of  Variables  Within  Groups: 
A  Hypothetical  Model 

Figure  5  presents  a  theoretical  model  of  the  relationships  of  what  possibly  could  be  the 
most  instrumental  variables  in  the  study.  While  results  of  the  study  indicated  correlations 
between  various  types  of  variables,  no  statistical  analyses  of  directionality  were  conducted. 
Therefore,  while  data  suggest  a  possible  tendency  of  movement  from  the  top  of  the  diagram 
downward,  and  from  right  to  left,  there  may  in  fact  be  circular  relationships  operating 
among  the  variables.  Also,  before  accepting  this  model,  not  only  cause-effect  relationships 
should  be  tested,  but  between- groups  tests  of  significance  should  be  conducted  on  the  unique 
correlations  in  each  group. 

The  treatment  evoked  different  associations  for  each  group  between  motivation  and 
learning  strategy  use,  which  may  be  related  to  differences  in  performance  directly,  or 
indirectly  (through  the  role  of  individual  characteristics  or  motivational  affects). 

In  addition,  learning  strategy  value  appears  to  be  the  variable  most  closely  associated 
with  training;  and  use  of  particular  learning  strategies  appears  to  be  the  variable  most 
closely  associated  with  performance.  Other,  possibly  intervening  and  interacting,  variables 
are  discussed  below. 

Learning  Strategy  Value.  Use  and  Individual  Characteristics 

Data  from  the  study  suggest  that,  because  the  treatment  group  associated  value  of 
learning  strategies  with  use  to  &  greater  degree  than  the  control  group,  they  may  have  used 
strategies  (including  primary  It  .  ning  strategies)  more  effectively.  As  evidenced  by  unique 
associations  with  personal  beliefs  and  attributions  (individual  characteristics),  the 
treatment  group  may  have  had  more  unconditional  acceptances  of  learning  strategy  use. 

Conversely,  the  control  group  may  have  viewrd  burning  strategies  as  generally  not 
helpful,  to  be  used  only  if  the  task  wuh  interesting,  it  was  convenient  or  easy  to  do  so,  or  as 
a  Inst  resort  to  avoid  failing. 
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Theoretical  Rnle  of  Individual  Characteristics  in  Outcomes 


Personal  interest  and  value  in  a  task  or  subject  may  be  key  individual  characteristics 
related  to  academic  performance,  motivation,  and  learning  strategy  use  (both  primary  and 
other  learning  strategies).  Results  indicate  that  the  control  group  had  more  of  a  tendency  to 
associate  the  use  of  a  strategy  with  task  interest,  effort,  and  unproductive  personal 
attributions  (i.e.,  the  need  to  avoid  academic  failure). 

Conversely,  results  indicate  that  the  treatment  group  had  more  of  a  tendency  to 
associate  overall  task  value  (interest,  importance,  and  utility)  with  learning  strategy  use.  It 
appears  that  the  treatment  planted  an  unconscious,  unconditional  recognition  of  the  value  of 
learning  strategy  use. 

In  addition,  follow-up  interviews  suggest  that  task  value  or  interest  may  have  the  power 
to  override  knowledge  of  either  subject  or  learning  strategies  as  an  academic  motivator. 

Figure  5 

It  also  appears  that  the  following  three  general  principles  may  be  at  work: 

1.  Individual  characteristics  (i.e.,  task  importance  and  utility)  may  affect  use  of 
"primary"  learning  strategies  (groups  of  cognitive  and  metacognitive  strategies),  which  nee  in 
to  be  directly  related  to 

motivational  outcomes,  and  indirectly  to  performance. 

2.  Individual  characteristics  may  affect  use  of  "other"  learning  strategies  (particular 
strategies  and  techniques,  including  resource  management)  that  may  directly  influent 
performance. 

3.  The  mix  of  individual  characteristics,  "primary"  (self- regulated)  learning  strategics, 
and  motivation  outcomes,  in  combination  with  particular  other  learning  strategics,  may 
indirectly  lead  to  different  performance  outcomes. 

The  study  did  not  assess  learning  strategy  goals,  yet  the  treatment  could  also  have 
affected  such  goals.  According  to  Weinstein  and  Mayer  (1986),  the  goal  of  any  particular 
learning  strategy  may  be  the  "...way  in  which  the  learner  selects,  acquires,  organizes,  or 
integrates  new  knowledge"  (p.  315).  Perhaps  the  treatment  affected  one  or  more  of  these 
variables. 

Individual  Characteristic  Motivation  and  Performance 

High  correlations  between  task  interest  and  motivation,  as  well  as  correlations  between 
self-regulating  strategies  (planning  and  regulating)  and  both  self-efficacy  and  expectancy  for 
success  may  have  evoked  other  learning  strategy  use  that  was  relevant  to  increased 
performance. 

Theoretical  Motivational  Patterim 

Results  confirmed  the  dynamism  of  c  ontinuing  intrinsic  motivation,  which  has  been 
defined  by  McCombs  (1984)  as: 
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a  dynamic,  internally  mediated  set  of  metacognitive,  cognitive,  and  affective  processes 
(including  expectations,  attitudes,  ar-*  beliefs  about  the  self  and  the  learning 
environment)  that  can  influence  a  student's  tendency  to  approach,  engage  in,  expend 
effort  in,  and  persist  in  learning  tasks  on  a  continuing  self-directed  basis  (p.  200). 

Data  from  the  study  suggest  that  self-efficacy  may  have  been  the  motivational  variable 
most  related  to  differences  between  the  groups  in  relationships  among  the  other  variables. 
For  example,  the  treatment  group  had  more  associations  with  self-efficacy  thar  the  control 
group,  including  the  previously  mentioned  links  with  learning  strategy  use. 

Secondly,  the  treatment  group  associated  self-efficacy  with  learning  strategy  use,  as 
opposed  to  the  control  group's  association  of  self-efficacy  with  performance.  Thirdly, 
although  there  were  no  significant  differences  in  scores,  the  treatment  group  showed  higher 
correlations  (synergistic  effects)  between  expectancy  for  success  and  desire  to  learn  (43 
percent  higher),  and  desire  to  learn  and  self-efficacy  (49  percent  higher). 

Although  self-efficacy  and  active  engagement  in  learning  are  seen  in  the  literature  as 
motivational  aspects  of  self- regulated  learning,  the  literature  provided  no  definitive 
descriptions  of  motivational  outcomes  of  self- regulated  learning.  What  was  discussed  is  a 
circular  relationship  between  achievement  and  motivation;  for  example,  Corno  (1986) 
implied  that  one  must  be  motivated  to  achieve,  yet  one  may  be  motivated  by  achievement. 

Follow-up  interviews  in  this  study  supported  the  idea  that  learning  strategy  use  can  he 
motivational,  and  the  treatment  group  did  have  higher  scores  on  two  performance 
(achievement)  outcomes. 

Pintrich  and  De  Groot  (1988)  suggest  that  important  factors  in  student  achievement 
include  planning  (among  other  metacognitive  strategics),  self-efficacy,  and  motivation  to 
employ  self* regulated  strategies  and  regulate  cognition  and  effort.  In  addition,  Easton  and 
Ginsberg  (1983)  and  Zimmerman  (1986)  note  that  planning  is  an  achievement  factor. 

As  related  to  this  study,  planning  significantly  mereUHed  for  the  pilot  subjects  (N  =  90). 
and  both  self-efficacy  and  expectancy  for  success  Hignificantly  increased  for  pre-pilot  subjects 
(N  =  20)  two  and  one-half  months  after  the  training  intervention,  during  which  time  they 
were  enrolled  in  a  course  involving  nothing  else  but  the  analysis  of  case  studies. 

It  is  theorized  that  perceived  self-efficacy  (a  combination  of  perceived  personal 
competence  and  self-confidence,  or  ability  to  discriminate  between  controllable  and 
uncontrollable  situations)  leads  to  increased  motivation  (Bandura,  1986;  McCombs,  1988). 
Perhaps  the  treatment  caused  a  link  between  self-efficacy  and  some  other  variable,  which 
produced  the  same  effect  as  self-efficacy  alone. 

Various  researchers  (Bandura,  1986;  McKeachie,  1988;  McCombs,  1988;  Schunk,  1987) 
indicate  that  self-efficacy  may  either  affect  learning  strategies  training  or  be  affected  by  it. 

Concerning  the  relationship  between  self-efficacy  and  performance,  Bandura  (1986) 
notes  that  people  who  have  high  self-efficacy  will  "mobilize  greater  effort  and  persist  longer 
on  a  task"  (p.  413).  In  this  connection,  self-efficacy  is  an  important  element  of  self -regulated 
learning  which,  when  combined  with  low  anxiety  toward  learning,  is  a  good  predictor  of 
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student  achievement,  according  to  Pintrich  and  De  Groot  (1988). 

However,  is  self-efficacy  necessary  for  performance,  or  is  it  an  effect  of  performance?  In 
analyzing  performance  results  for  this  study,  it  appears  that,  for  the  treatment  group,  self- 
efficacy  may  have  resulted  from  a  knowledge  of  learning  strategy  use  (or  other  related 
variables)  and  then  become  a  precursor  to  performance.  Conversely,  it  appears  that,  for  the 
control  group,  lower  self-efficacy  was  a  result  of,  instead  of  a  precursor  to,  performance,  since 
the  control  group  subjects  did  not  have  a  storehouse  of  learning  strategies  presented  to  them 
in  the  study. 

Also,  since  self-efficacy  and  expectancy  for  success  are  highly  correlated,  it  might  be  said 
that,  because  the  treatment  group  had  training  in  learning  strategies,  they  may  have  had 
higher  expectations  for  success  and,  therefore,  higher  self-efficacy,  despite  insignificant  higher 
raw  scores  with  large  standard  deviations. 

Regarding  the  interaction  of  motivation  and  self- regulated  learning,  if  one  agrees  that  the 
functions  of  learning  strategies  training/use  are  both  cognitive  and  affective,  and  involve  self- 
directed  learning  (Weinstein  &  Mayer,  1986;  McCombs,  1987),  it  is  obvious  that  the 
experimental  treatment  made  a  difference.  Furthermore,  if  one  accepts  McCombs'  (1987) 
argument  that  all  learning  is  inherently  autonomous  (independent  and  self-directed)  and 
that  students  ultimately  have  to  be  guided  by  their  own  motivation  (including  interests)  and 
ability,  it  would  appear  that  the  treatment  group  exhibited  more  autonomous-like 
interactions. 

Implicates  for  Practice 

Implications  for  practice  include  the  integration,  by  both  educators  and  students,  of  task- 
related  learning  strategies  into  educational  programs  at  all  levels.  The  instructional 
package  would  include  knowledge  of  subject  content,  learning  strategies,  and  attributions 
awareness  (knowledge  of  how  individual  characteristics,  values,  and  motivation  may  affect 
learning). 

Recommendations  for  Research 
Recommendations  for  further  research  are  listed  below. 

1 .  Study  synergistic  interactions  of  more  than  one  variable  at  a  time  when  analyzing 
perform0 nee  and  motivational  outcomes. 

2.  Regarding  motivational  variables,  continue  to  assess  self-efficacy  and  exroctancy  for 
success  in  the  self-concept  context.  Desire  to  learn  is  a  more  complex  variable  in  that  it 
incorporates  more  than  the  self-concept  context.  An  additional  variable  would  be  effort. 

3.  Regarding  instrumentation,  subjects  need  to  be  tested  as  to  their  actual  knowledge 
and  application  of  learning  strategies.  Such  assessments  as  journals,  observation  and 
testing  might  be  appropriate. 

4.  Allow  adequate  time-i.e.,  a  minimum  of  three  to  six  months-for  appropriate  practice, 
mastery,  and  motivation  for  actual  use  of  learning  strategies. 

5.  Incorporate  peer/teacher  discussion  in  the  learning  strategies  package. 
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fi.  Via  cluster  iiiuilvHiK,  tlf "tt-riiiim*  if  students  fall  into  particular  categories  of  learning 
strategics  use  and  unit  ivut  icimil  characteristics. 

7.  Determine  which  particular  strategics  cause  the  most  variance  in  particular 
performance  and  motivation  outcomes. 

H.  Determine  whether  nine  guidelines  act  as  strategies  by  setting  up  a  control  group 
which  in  not  provided  with  such  extensive  instructions. 

9.  Complete  move  classroom  application  studies  similar  to  this  study;  but  use  more 
complex  problems  and  add  problem  solution  to  problem  statement  as  a  performance 
outcome. 

10.  Integrate  learning  strategies  training  with  other  types  of  tasks  and  compare  results 
of  training. 

11.  Apply  findings  to  not  only  other  subject  areas, '  \t  all  levels  and  types  of  education- 
elementary,  secondary,  postsecondary,  and  adult  and  on-the-job  training. 

12.  Test  the  proposed  directional  associations  in  the  theoretical  model  concerning  the 
relationships  between  learning  strategies  training,  learning  strategics  valued,  individual 
characteristics,  self-regulated  learning,  other  learning  strategies  used,  and  performance, 

1  ;J.  Continue  developing  theoretical  models  that  portray  relat  Kinships  between 
significant  variables  that  impact  on  motivation  and  performance.  This  involves  reviewing 
literature  in  the  various  areas  of  motivation,  metacognition,  self  regulated  learning,  learning 
strategies,  instructional  design,  and  higher  education,  etc. 
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Appendix  A 


Operational  Dufinittona 


Included  herein  are  operational  definitions  of  variables  studied.  They  are  categorized  by 
performance  outcomes,  motivational  outcomes,  individual  characteristics,  learning  strategies 
abilities  and  self-regulated  learning,  and  learning  strategies  training.  All  assessments 
except  performance  were  based  on  self-report. 

Performance  Outcomes 

Performance  includes  separate  scores  on  three  different  aspects  of  a  case  problem 
analysis:  (a)  Listing  symptoms  that  indicate  management  problems  exist;  (b)  classifying 
symptoms  as  appropriate  management  functions  (planning,  organizing,  directing,  and 
controlling)  and/or  processes  (decision  making  and  communication). 

Motivational  Outcomes 

Since  motivation  is,  by  definition,  consciously  or  unconsciously  decided  goal-directed 
behavior,  goal  orientation  and  self-concept  motivational  variables  are  emphasized  as 
outcomes  in  this  study~i.e.,  desire  to  continue  learning,  expectancy  for  success,  and  self- 
efficacy  for  learning. 

Desire  to  continue  learning.  Desire  to  continue  learning  includes  the  desire  to  learn  more 
about  both  management  in  general  and  case  study  analysis  in  particular. 

 Expectancy  for  success.  Expectancy  for  success  relates  to  expectations  for  task-specific 

academic  success.  It  indicates  generally  "how  well"  and,  in  particular,  the  extent  to  which 
subjects  will  obtain  high  scores  on  business  case  problem  tasks. 

Self- efficacy  for  learning.  Self-efficacy  for  learning  is  operntiunalixecl  as  perceived  task- 
related  self-efficacy  for  learning.  Components  are  understanding  difficult  written  or  oral 
information  presented,  learning  basic  concepts,  mastering  skills  being  taught,  and 
confidence  in  completing  assignments  and  tests  involved  in  a  case  problem  research  project. 

Individual  Characteristics 

 Twenty-two  individual  characteristics  were  assessed.  Ten  individual  characteristics  are 

defined.  An  additional  12  demographic  variables  are  simply  listed  since  they  are  self- 
explanatory. 

Ability  beliefs.  These  reflect  the  causal  role  a  student  believes  ability  plays  in  obtaining 
good  grades  and  understanding  material  presented  at  school. 

Attributions  to  failure  experiences.  These  reflect  beliefs  about  the  causal  role  four 
elements  (ability,  context,  effort  and  ability)  play  in  such  assessments  of  "academic  low 
points"  as  getting  low  grades,  not  understanding  school  materials,  or  not  doing  as  well  as 
expected. 
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Attributions  to  success  experiences.  These  reflect  beliefs  about  the  causal  role  four 
elements  (ability,  context,  effort  and  ability)  play  in  such  assessments  of  academic  success 
as  good  grades,  overcoming  obstacles  in  the  path  of  academic  success,  having  the  teacher 
think  highly  of  one's  work,  and  success  on  exams. 

Context  beliefs.  These  beliefs  account  for  the  assignment  of  causality  for  academic 
success  or  failure  to  such  contextual  elements  as  the  teacher's  grading  scheme,  difficulty  level 
of  course,  teacher's  opinion  of  student,  or  teacher's  ability  to  make  course  interesting. 

Demographic  variables.  These  included:  Ethnic  group,  gender,  degree  program,  college 
major,  class  level,  completion  of  an  Introduction  to  Business  course,  interest  in  participating 
in  study,  years  since  high  school  graduation,  number  of  years  full-time  work  experience, 
number  of  current  semester  credit  hours,  total  credit  hours  completed,  and  cumulative 
gradepoint  average. 

Effort  beliefs.  These  beliefs  account  for  the  assignment  of  causality  for  academic  success 
or  failure  to  personal  effort  or  degree  of  studying  or  working  hard. 

External  causality  beliefs.  These  beliefs  assign  causality  for  academic  success  or  failure 
to  luck  or  contextual  factors  listed  in  "context  beliefs"  (i.e.,  the  teacher's  actions  or  the  course 
itself)* 

Internal  causality  beliefs.  These  beliefs  assign  causality  for  academic  success  or  failure 
to  personal  ability  and  effort. 

Luck  beliefs.  These  beliefs  assign  causality  for  academic  success  or  failure  to  luck-i.e,, 
having  the  right  questions  show  up  on  an  exam  or  getting  had  breaks. 

Task  value.  Task  value  incorporates  three  aspects:  (a)  interest  in  the  subject  matter  of 
the  course,  (b)  the  importance  of  understanding  management  rase  problems,  and  (c)  the 
usefulness  (utility)  of  learning  about  case  problem  analysis, 

Test  anxiety.  Test  anxiety  incorporates  two  aspects,  cognitive  interference  and 
emotionality.  Cognitive  interference  is  the  degree  to  whtah  students  think  about  items  on  a 
test  they  can't  answer,  the  consequences  of  failing,  and  how  poorly  they  are  doing  in 
comparison  with  other  students.  Emotionality  refers  to  the  uneasy,  upset  feelings  (i.e.,  fast- 
heat  mg  heart)  that  may  arise  in  taking  an  exam, 

Luirning  Strategics  Abilities  and  Self-Regulated  Learning 

I 'filming  strategies  are  flexible  patterns  of  thoughts  and  behaviors  used  to  acquire, 
manipulate,  and  retain  knowledge  and  skills.  Three  major  categories  of  learning  strategies 
are  cognitive,  metacognitive,  and  resource  management  strategies.  The  cognitive  aspect  is 
knowledge  about  particular  strategies;  the  metacognitive  aspect  is  ability  to  plan  for  anil 
regulate  strategy  use.  Resource  management  strategies  relate  to  the  use  of  external  sources 
of  learning  support  and  include,  in  this  study;  Timing  work,  asking  others  for  help,  and 
concent  rating  in  a  quiet  place. 

1  lowcver,  knowledge  of  learning  strategies  alone  is  not  enough  to  affect  performance  fitul 
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motivational  outcomes.  Active  control  over  and  engagement  in  the1  learning  process  through 
application  of  learning  strategies  is  needed.  This  self-controlled  use  or  application  of  learning 
strategies  is  known  as  self-regulated  learning. 


Learning  Strategies  Training 

In  this  study,  a  variety  of  learning  strategies  were  provided  in  the  training  treatment. 
Consequently,  two  categories  of  learning  strategy  use  were  assessed:  (u)  "self-regulated"  or 
"primary"  learning  strategies,  and  (b)  "other"  learning  strategies, 

The  assessment  of  "primary"  strategies  followed  Pintrich  et  al's  (1989)  assessments  of 
four  cutegories  of  cognitive/metacognitive  strategies  related  to  self-regulated  learning: 
Organization,  Elaboration,  Planning  and  Regulating.  These  strategics  are  defined  below, 
per  Pintrich  etal  (1989). 

The  assessment  of  "other"  learning  strategies  involved  the  reported  use  of  twelve 
particular  Htudy  techniques,  rather  than  groups  of  strategies,  that  had  been  listed  as  being 
useful  in  romp  luting  the  ease  study  tasks  in  the  pilot  study.  These  other  learning  learning 
Ht  rut  ogles  are  iiIho  list  ed  below;  they  included  organizing,  regulating,  and  resource 
management  techniques,  as  indicated  in  parentheses. 

tiluburut-liin  itmlmtiuM'  Elaboration,  as  related  to  business  case  studies,  includes  a 
composite  of:  (a)  pulling  together  information  from  different  sources,  such  as  lectures, 
handout  Information  or  readings;  (h)  trying  to  relate  new  material  to  what  the  subject 
already  knows;  (e)  writing  brief  summaries  of  main  ideas  in  a  case;  (d)  making  connections 
between  readings  and  concepts  from  lecture;  (e)  applying  ideas  from  other  classes  to  cases; 
ami  (i)  relat  ing  ideas  learned  from  management  case  studies  to  other  courses. 

prganizaLiun  strategies.  Cast1' related  organization  strategies  include  a  composite  of:  (a) 
locating  important  ideas;  (b)  outlining  both  case  study  materials  and  personal  notes;  and  (c) 
making  simple  charts,  diagrams  or  tables  to  organize  or  summarize  materials. 

planning  strategies,  ('use-related  planning  strategies  include  a  composite  of:  (a) 
skimming  new  material  before  studying  it  thoroughly;  (b)  making  up  questions  about 
written  materials;  (e)  setting  goals  to  direct  activities  in  each  study  period;  and  (d)  thinking 
through  what  is  to  be  learned  rather  than  just  reading  written  material. 

Itumilillim  strategies.  Case-related  regulating  strategies  Include:  (a)  Rereading 
confusing  material  to  figure  it  out;  (b)  changing  the  way  difficult  material  is  read;  (c) 
adapting  studying  and  learning  style  to  instructor's  requirements  and  teaching  style;  and  (d) 
sorting  out  confusing  ideas  or  instructions  and  doing  stunothing  about  it  before  next  class 
meeting. 

Other  learning  strategics.  Other  learning  strategies  Included  individual  (rather  than 
composite)  assessments  of  use  (LSU)  and  value  (LHV)  of  the  following  twelve  strategies:  (1) 
having  all  materials  organized  before  beginning  task  (organizing);  (2)  timing  work  (resource 
management);  (tt)  asking  others  for  help  (resource  management);  (4)  reading  the  ease  more 
than  once  (regulating);  (5)  underlining  or  highlighting  important  points  (organizing);  (ft) 
writing  notes  in  the  margin  of  the  case*  (organizing);  (7)  taking  written  notes  about  the  case 
(organizing);  (8)  making  a  diagram  of  the  case  facts  (organizing);  (H)  using  a  worksheet  to 
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organize  symptoms  (organizing);  (10)  writing  a  rough  draft  of  problem  statement 
(organizing);  (11)  concentrating  in  a  quiet  place  (resource  management);  and  (12)  referring 
frequently  to  instructions  and  other  materials. 

Appendix  B 
Tables  1-11 

Tabic  1 

ik!umiuJLIuUL!ii^'LLuud  JJii  tf mimima     Uxperinmntal  Group  Differences:  Group  Ts  Unique 

mJiighia: 

<JiIXUiuUiaiiJ  hutwuyn  Motivation  UlUuWlia  (Hulf  Eflkacy,  Expectancy  for  Success,  and  Desire 

tu  LimnU 

and  Suit  Key ub lud  {Primary)  Learning  Htrutuyica 


Variables  Correlated 


Group  S 


Group  T 


CODE  A: 


Self-efficacy  and  Organization 
Self-efficacy  and  Elaboration 
Self-efficacy  and  Planning 
Self-efficacy  and  Regulatin 
Self-efficacy  and  Total  Metacognition 

(1()1)K  It: 

Kxportimry  lor  Sum»HH  unci  Klahoration 
Kxpivtimry  lor  HurvenH  and  Planning 
Mx|M»rtnnry  for  HntWHH  and  Tiitnl  Metacognition 

1'IHIK  T: 

DphIii*  In  I, earn  and  Organization 
DoHtrn  In  l«<*ilt'ti  and  Klahoration 
Di«hIi'i»  tu  \ivnvn  and  Planning 
DphIi'ii  In  Lcurn  and  Ueguhting 
Di'Hirr  to  Loam  and  Total  ietacognition 

COOK  J: 

Si'lf -i*ffli*jirv  and  tlrnirt*  to  loam 


.4294 
.5633 


.5315 


.5410 
.6094 
.6087 
,6261 
.7049 


.4281 


.5135 
.7801 
.6204 
,6677 
,6855 


,7345 
.6690 
.6490 


.6215 
.7029 
.7320 
,6937 
.7687 


.6114 
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Expectancy  tor  success  and  desire  to  learn  .3852  .6750 

Nptes:  xi  <  .06  for  correlations  within  each  group;  kuiwcen-group  significance  was  not 
calculated. 

N  for  total  sample  =  54;  N  for  Group  S  =  28;  N  for  Group  T  =  2« 
Table  2 

Figure  Code  D.  Summary  of  Experimental  Group  Differences:  luiquu  or  Higher  Correlations 
between 

Individual  Characteristics  and  Self-Regulated  (Primary)  UmniinF  fitrattigitiB 


INDIVIDUAL  CHARACTERISTICS  CORRELATED  WITH  ORGANIZATION  STRATEGIES: 
Group  S  -  None 

Group  T  -  Importance  of  task,  utility  of  task,  overall  tusk  value  (interest,  importance  and 
utility),  negative  context  beliefs,  negative  externality  causality,  and  negative  overall 
externality  causality  beliefs  (All  unique  with  range  =  .4301  to  .6176) 

INDIVIDUAL  CHARACTERISTICS  CORRELATED  WITH  ELABORATION  STRATEGIES: 

Group  S  •  None 

Group  T  -  Unique  (ranging  from  .4435  to  .5075)  -  Interest  in  task,  negative  context  beliefs, 
and  negative  externality  causality  beliefs.  Higher  •  importance  of  task  (,5964  vs.  ,3935)  and 
overall  task  value  (.6104  vs.  .4070). 

INDIVIDUAL  CHARACTERISTICS  CORRELATED  WITH  PLANNING  STRATEGIES: 
Group  S  -  Task  utility  (.4090) 

Group  T  -  Unique  (ranging  from  .4158  to  .5778)  -  Interest*  in  task,  importance  of  task,  and 
negative  relationships  with  context  beliefs,  luck  beliefs,  externality  causality,  overall 
externality  causality  beliefs,  attributions  to  success  experiences,  and  attributions  to  failure 
experiences.  Higher  •  Task  value  (.5452  vs.  .4326) 

INDIVIDUAL  CHARACTERISTICS  CORRELATED  WITH  REGULATING  STRATEGIES: 
GroupS -Effort  (,4424) 

Group  T  -  Unique  (.401 1  to  .5978  range)  •  Interest  in  task,  years  since  high  school,  and 
negative  relationships  with  test  anxiety  (cognitive  interference),  context  beliefs,  luck  beliefs, 
externality  causality  beliefs,  and  attributions  to  failure  experiences.  Higher  •  Importance  of 
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task  (.6104  vl.  .4022)  and  overall  task  value  (.6291  vs.  .3971) 


Notes:  This  material  has  boon  extracted  from  Table  24,  Nuttall  (1991).  £  <  .05  for 
correlations  within  each  group;  between-group  significance  was  not  calculated.  Total  sample 
N  =  54;  N  for  Group  S  =  28;  N  for  Group  T  =  26. 

Table  3 

Figure  Codes  E.G.  Summitry  of  Knporimcutal  Group  Differences-  Unique  or  Higher 
Correlations 

between  Motivation  Outtuuiios  fHelf-Kffieacv.  Knpeeiancv  for  Success,  and  Desire  to  Learnt 
and 

individual  Quuattdattialkti 


Variables  Corr.lated  Group  S  Group  T 


CODE  K  (Group  T  ia  unique  jr  higher); 

Self- efficacy  ar.o  task  interest 
Self-efficacy  and  task  utility 
Self-efficacy  and  overall  task  value 
Self-efficacy  and  'uck  beliefs 
Self-efficacy  and  external  causality  beliefs 

CODE  F  (Both  groups  have  unique  correlations): 


Expectancy  ior  success  and  task  importance  .433 1 

Expectancy  for  success  and  task  interest  -  .6308 

Expectancy  for  success  and  luck  beliefs  -                 -.496  J 

Expectancy  tor  success  and  external  causality  beliefs  -                -  A 1 7 1 

CODE  G  (Eoth  groups  have  unique  correlations): 

Desire  to  learn  and  task  utility  .4765 

Desire  to  learn  and  effort  beliefs  .4866 

Desire  to  learn  and  internal  beliefs  .3949 

Desire  to  learn  and  task  interest  .4515  .7153 

Desire  tc  '  am  and  task  importance  .4993  .5961 

Desire  to  learn  and  overall  task  value  .5594  .6759 

Desire  to  learn  and  context  beliefs  -.5081 

Desire  to  learn  and  luck  beliefs  -.4197  ' 

Desire  to  learn  and  external  caur.ality  beliefs  •  -.5085 

Desire  to  learn  and  attributions  to  failure  experiences  -.5306 


.4046 


.677" 
.4739 
.6756 
-.4145 
-.4045 
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Desire  to  learn  and  years  since  high  school  graduation 


.5411 


iJotfia:  £  <  .05  for  correlations  within  each  group;  between-group  significance  was  not 
cplculated. 

Total  sample  N  =  54;  N  for  Group  S  =  28;  N  for  Group  T  =  26, 
Table  4 

Figure  Codes  H.  I.  and  R-S.  Summary  of  Esgperi mental  Group  Differences:  Unique  or  Higher 

Correlations  between  Performance  (Symptoms.  Functionsr  and  Problem)  and  Other 
Variables 


Variables  Correlated 


Group  S  Group  T 


CODE  H  (Group  T  is  unique): 

Symptoms  and  valuing  asking  others  for  help  (LSV3) 


.5588 


CODE  I  (Both  groups  have  unique  correlations): 

Problem  and  luck  beliefs  -.4418 
Problem  and  semester  credit  hours  -.4 1 85 

Problem  and  female  gender 
CODE  R  (Group  S  is  unique): 

Functions  and  self -efficacy  .4744 
CODE  S  (Group  S  is  unique): 

Problem  and  concentrating  in  a  quiet  place  (LSU11)  .4408 
CODE  T  (Group  S  is  unique): 

Problem  and  valuing  asking  others  for  help  (LSV3)  -.3981 
Problem  and  valuing  concentrating  in  a  quiet  place  (LSV1 1)  .5825 


.4694 


Notes:  £  <  .05  for  correlations  within  each  group;  hetween-group  significance  was  not 
calculated. 

Total  sample  N  =  54;  N  for  Group  S  =  28;  N  for  Group  T  =  26. 
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Table  5 

Figure  Codes  L.  U  .  V.  and  Q*.  Summary  of  Experimental  Group  Differences:  Unique  or 

Correlations  between  Motivation  (Self- Efficacy.  Expectancy  for  Success,  and  Desire  to  Learnt 
and  Other 

Learning  Strategies  Used 


Variables  Correlated  Group  S  Group  T 


COD  K  L  (Both  groups  have  unique  correlations): 

Self-efficacy  and  timing  work  .4466 
Self-efficacy  and  making  a  diagram  of  case  facts  .5151 
Self-efficacy  and  writing  notes  in  margin  .3922 
Self-efficacy  and  referring  frequently  to 
inntructions  and  other  materials  .5199 

CODK  U  (Group  S  has  unique  correlations): 

Expectancy  for  success  and  timing  work  .3754 
Expectancy  for  success  and  making  a  diagram  of  case  facts  .5292 

CODK  V  (Group  S  has  unique  correlations): 

Desire  to  learn  and  having  all  materials  organized 

before  beginning  task  .5480 

Desire  to  learn  and  reading  the  case  more  than  once  .5850 

Desire  to  k*arn  and  concentrating  in  a  quiet  place  .4737 

Desire  to  learn  and  referring  frequently  to  instructions  and 

other  materials  .4133 


Figure  CODE  Q  indicates  that  Group  T  exhibited  higher  correlations  between  other 
learning  strategy  use  and  value  (See  Tables  10  and  11),  with  one  exception:  Group  S's 
correlation  of  use  with  value  was  higher  for  referring  more  frequently  to  instructions  and 
other  materials  (.7080  vs.  .5467). 

Notes:  £  <  .05  for  correlations  within  each  group;  bet  ween -group  nignificanco  was  not 
calculated. 

N  for  total  sample  =  54;  N  for  f  I  roup  H  ~  2H;  N  lor  t  Imup  T  -  2K. 
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Figure  CuAvh  MO.  Summary  of  Experimental  Group  Differences:  Unique  or  Higher 
Imtwuen  Motivation  (gdLEfficacy.  Expectancy  for  Success,  and  Desire  to  Learn)  and  Other 

StmtL'mta  Valued 


Variables  Correlated 


Group  S  Group  T 


CODE  M  (Both  ^oups  have  unique  correlations): 

Self-efficacy  and  valuing  timing  work  .4666 
Self-efficacy  and  valuing  making  a  diagram  of  case  facts  .5347 
Self-efficacy  and  valuing  having  all  materials  organized 

before  beginning  task  -  .4124 

Self-efficacy  and  valuing  referring  frequently  to  instructions 

and  other  materials  -   .  .4872 

Self-efficacy  and  valuing  underlining  or  highlighting  important 

points  -  .4362 

CODE  N  (Both  groups  have  unique  correlations): 

Expectancy  for  success  and  valuing  making  a  diagram  of  case  facts  .4242 
Expectancy  for  success  and  valuing  having  all  materials  organized 

before  beginning  task  -  .4377 

Expectancy  for  success  and  valuing  underlining  or  highlighting 

important  points   .4267 

Expectancy  for  success  and  valuing  referring  frequently  to 

instructions  and  other  materials  *  .4872 

CODE  O  (Both  groups  have  unique  correlations): 

Desire  to  learn  and  valuing  having  all  materials  organized  before 
beginning  task  .5253 

Desire  to  learn  and  valuing  reading  the  case  more  than  once  .5057 

Desire  to  learn  and  valuing  concentrating  in  a  quiet  place  .4598 

Desire  to  learn  and  valuing  using  a  worksheet  to  organize  symptoms  ,4410 

Desire  to  learn  and  valuing  referring  frequently  to  instructions 
and  other  material  .4850 

Nol£&:  p.  <  .05  for  correlations  within  each  group;  between -group  significance  was  not 

calculated. 

Total  sample  N  =  54;  N  for  Group  S  =  28;  N  for  Group  T  =  26. 
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Table  7 

Figure  Cede  l\  Summary  ui  LxjjcnuiuiUl-Umuy  DilTerencgg;  Group  £a  Unique  Correlations 
between 


Othpr  Learnititf  StriitotfiuH  UuluI  and  Individim^Wacteristics 

Vnriiilili'H  Correlated  Group  S 

Task  interest  and: 

(a)  asking  others  lor  help  (LKIJU)  .3869 

(b)  reading  the  case  more  than  mire  (LSU4)  .4291 

Test  anxiety  (emotionality)  and  timing  work  (LSU2)  .4070 

Overall  test  anxiety  and  timing  work  (I,SU2)  .3867 
Effort  beliefs  and: 

(a)  having  all  materials  organized  before  beginning  task  (LSU1)  4554 

(b)  timing  work  (LSU2)  .5122 

(c)  reading  the  case  more  than  once  (LSU4)  .6523 

(d)  referring  frequently  to  instructions  and  other  materials  (LSU12)  .6020 

Internal  causality  beliefs  and: 

(a)  having  all  materials  organized  before  beginning  task  (LSU1)  3864 

(b)  reading  the  case  more  than  once  (LSU4)  .5255 

(c)  underlining  or  highlighting  important  points  (LSU5)  .4262 

(d)  referring  frequently  to  instructions  and  other  materials  (LSI)  12)  .5154 

External  beliefs  and  timing  work  (LSU2)  .4565 

Attributions  to  success  experiences  and: 

<a)  having  all  materials  organized  before  beginning  task  (LSU1)  4578 

(b)  timing  work  (LSU2)  .5063 

(c)  reading  the  case  more  than  once  (LSU4)  .5171 

Attributions  to  failure  experiences  and  timing  work  (LSU2)  .4266 


Notes:  £  <  .05  for  correlations  within  each  group;  between -group  significance  was  not 
calculated. 

N  for  total  sample  =  54;  N  for  Group  S  =  28;  N  for  U roup  T  =  2(i 
Table  8 

Figure  Code  P.  Summary  of  Experimental  Group  Differences:  Group  S's  Unique  Correlations 

between 

Other  Learning  Strategies  Used  and  Demographics 

Variables  Correlated  ( Sroup  S 


Negative  relationship  between  Introduction  to  Business  rnurse  and: 

(a)  timing  work  (LSU2) 

(b)  taking  written  notes  about  the  case  (LSU7) 


.4568 
.4217 


Negative  relationship  between  interest  in  study  and: 

(a)  having  all  materials  organized  before  beginning  task  (LtiUl)  MtMfi 

(b)  reading  the  case  more  than  once  (LSU4)  .  3H V 1 

Male  gender  and  underlining  or  highlighting  important  point h  (LSI! 5)  4074 
Negative  relationship  between  semester  credit  hours  and: 

(a)  writing  notes  in  the  margin  of  the  case  (LSU6)  .  3H 1 1 

(b)  concentrating  in  a  quiet  place  (LSUll)  .fWlfi  1 

Negative  relationship  between  GPA  and: 

(a)  timing  work  (LSU2)  AMM 

(b)  asking  others  for  help  (LSU3)  .  6080 

Negative  relationship  between  absences  and  making  a  diagram  of 

the  case  facts  (LSU8)  . 4 


*There  were  no  significant  main  effects  for  gender  at  the  .05  level  for  using  this  strategy. 

Notes:  n  <  .05  for  correlations  within  each  group;  between-group  significance  was  not 
calculated. 

N  for  total  sample  =  54;  N  for  Group  S  =  28;  N  for  Group  T  =  26 
Table  9 

Figure  Code  P.  Summary  of  Kxpurimontal  Group  Differences:  Group  T's  Unique  Correlations 
between 

Other  Learning  Strategics  Used  and  Individual  Characteristics/Demographics 


Vitriulih'H  Correlated  Group  T 

Task  utility  and: 

(a)  having  all  materials  organized  hvtorv  beginning  task  (LSU1)  AWM 

(b)  using  a  worksheet  to  organize  symptoms  (LSU9)  .4101 

(c)  referring  frequently  to  instructions  and  other  materials  (LSUl  2)  .H.'ifi2 

Task  importance  and: 

(a)  using  a  worksheet  to  organize  symptoms  (LSU9)  .6839 

(b)  referring  frequently  to  instructions  and  other  materials  (LSU12)  .4704 

Overall  task  value  (interest,  importance,  and  utility)  and: 

(a)  using  a  worksheet  to  organize  symptoms  (LSU9)  .4829 

(b)  referring  frequently  to  instructions  and  other  materials  (LSU12)  .6232 

Test  anxiety  (emotionality)  and  making  a  diagram  of  the  case  facts  (LSU8)  .3890 

Attributions  to  success  experience  and  writing  a  rough  draft  of  problem 

statement  (LSU 10)  .4241 

Associate  degree  major  (vs.  bachelor's)  and  timing  work  (LSU2)  .4799 


Notes:  p  <  .05  for  correlations  within  each  group;  between-group  significance. was  not 
calculated. 
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N  for  total  sample  =  54;  N  for  Group  S  =  28;  N  for  Group  T  =  26 
Table  10 

Correlations  of  Other  Learning  Strategies  Valued  (LSVl-6^  with  Other  Learning  Strategies 
Used  for 


Total  Sample  and  Experimental  Group 
Strategies  Used  LSVl 


Strategies  Valued 
LSV2       LSV3  LSV4       LSV5  LSV6 


LSUl 


LSU2 


I, sua 


LSUl 


LSI  15 


LSU(i 


LSUH 


LSU9 


LSUll 


LSUl  2 


.8452 
.86  4  OS 
.7797T 


.4104S 


anr»7S 


aii'H 

.•HI  MS 

aHi2 
iirjas 


.5057 

.4408S 

.5314T 

.3258 
.5592S 


.8335  .3933 
.7659S 

.HttllT  .4940T 

.-AMYA  .7831 

.75498 
.(1 1  'JOT  .H.1R3T 


.4364 

.4136S 

.4619T 

.2925 
.4221T 


.3161 


.5179S 


.3131 


.4913 

.5730S 

.4002T 


.5083 
.6135S 

.3691  .6836 
.5558S 
..9068T 

.8457 
.7200S 

.4002T     .4320T  .9595T 


.4318T 

Notes;  d.  =  <  .05.  A  complete  listing  of  LSU/LSV  is  in  Exhibit  A.  N  for  total  sample  =  54; 
N  for  Group  S  =  28;  N  for  Group  T  =  26 
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Table  1 1 


Correlations  of  Other  Learning  Strategies  Valued  (LSV7-12^  with  Other  Learning  Strategies 
Used  for  Total  Sample  and  Experimental  Oroup 

Strategies  Valued 


Strategies  Used  LSV7 
LSU1 


LSU2 

LSU3 

LSU4 
LSU5 
LSU7 

LSU8 

LSU9 

LSU10 

LSU11 
LSU12 


.2977 


.3022 
.4274T 


.8196 

.6971S 

.9323T 


.4343 
.6388T 


LSV8  LSV9 

.3452 
.4112T 

.3703 
.3800S 


.8615  .3476 
.8707S  .5579S 
.8243T 

.3547  .8748 
.5933S  .8292S 
.9466T 


.4067T 


LSV10  LSVU 
.2676 

.2975 
.3091 


.3227 
6553T 
.3659 
.5809T 


.8680 

.7681S 

.9755T 


.423  IS 
.2714 


.7662 

.7895S 

.7889T 


.5467T 


LSV12 

.3250 
.4411T 


.4088 
.4762S 


.6568 
.7080S 


Notes:  p.  =  <  .05.  A  complete  listing  of  these  items  is  in  Exhibit  A.  N  for  total  sample  =  54; 
N  for  Group  S  =  28;  N  for  Croup  T  =  26 
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Ill  thiH  paper,  we  try  to  drstTihe  in  detail  the  characteristics  and  elements  of  situated 
learning.  We  then  describe  some  of  the  manifestations  of  instruction  that  are  intended  to 
exploit  this  perspective.  Strengths  and  weaknesses  of  these  perspectives  will  be  described 
in  the  context  of  the  data  presented  in  support  of  situated  learning  theories.  While 
providing  provocative  alternatives  to  current  conceptions  of  learning,  we  conclude  that  it  may 
not  be  possible  to  integrate  notions  of  Hit  it  lit  ed  learning  into  instructional  design  as  currently 
practiced. 

Situated  Cognition:  Alternative  Conceptions  of  Learning 

Situated  cognition  proposes  a  radically  different  explanation  of  learning,  conceiving  it 
as  a  largely  social  phenomenon.  Rather  than  occurring  within  the  mind  of  the  individual, 
learning  is  instead  described  hh  a  characteristic  of  many  social  interactions  that  take  place 
within  a  framework  of  participation  (Hanks,  1992).  Increasing  participation  in 
"communities  of  practice"  has  the  effect  of  engaging  the  whole  person,  focusing  on  "ways  in 
which  it  [learning]  is  an  evolving,  continuously  renewed  set  of  relations"  (Lave  &  Wenger, 
1991;  p.  50).  Indeed,  from  the  perspective  of  situated  cognition,  learning  requires  a  rich 
repertoire  of  essential  actors  and  participatory  relationships  beyond  those  common  to 
education  and  training  as  now  practiced.  Situated  cognition  also  proposes  a  different 
location  for  knowledge,  and  a  different  philosophy  of  knowledge  in  the  learning  process, 
along  with  an  emphasis  on  legitimate  peripheral  participation  in  social  groups  that  is 
characteristic  of  learners  in  a  variety  of  settings  and  cultures. 

Situated  action 

Theories  of  situated  cognition  take  the  view  that  human  activity  is  complex,  involving 
social,  physical  and  cognitive  factors.  Proponents  of  these  theories  believe  that  rather  than 
acting  on  symbolic  representations  of  the  world  that  are  located  in  the  mind,  we  are  in  direct 
contact  with  the  environment.  Cognitive  representations  only  become  necessary  when 
normal,,  situated  activity  fails  (Dreyfus  &  Dreyfus,  1986;  Greeno,  1989;  Winograd  &  Flores, 
1986).  Suchman  (1987)  describes  two  types  of  activity  in  which  humans  engage:  ad  hoc 
improvisation  during  action  and  representations  of  action  as  plans  or  retrospective  accounts. 
Situated  action  is  not  driven  by  plans,  or  in  other  words,  humans  "do  not  anticipate 
alternative  courses  of  action,  or  their  consequences,  until  some  course  of  action  is  already 
underway"  (Suchman,  1987,  p.  52). 

For  example,  suppose  we  wanted  to  build  a  robot  to  navigate  the  rocks,  boulders,  and 
craters  of  Mars.  We  have  two  basic  approaches.  In  the  first  approach,  we  can  build  a  robot 
with  an  "intelligence"  based  on  the  computational  metaphor  of  the  mind.  The  robot  would 
utilize  video  input  devices  to  view  the  terrain,  convert  the  input  into  symbols  used  by  the  on- 
board computer  to  recognize  patterns  in  the  terrain,  then  carefully  compute  plans  about 
where  to  move  next  (e.  g.,  which  leg,  how  far,  what  angle,  etc.).  In  the  second  approach,  we 
could  build  a  robot  that  had  "mini-intelligent"  legs  and  arms  that  could  automatically  sense 
the  changes  in  terrain  and  adjust  to  the  undulations  each  leg  might  encounter.  Its  actions 
would  be  situated  in  the  context  of  walking  on  Mars,  and  it  would  only  need  to  stop  and 
"plan"  if  it  encountered  a  vertical  wall,  a  canyon  or  some  other  obstacle  that  caused  its 
"mindless"  functioning  to  stop.  The  outcome  of  purposeful,  goal-directed  behavior  has 
seemingly  been  achieved  by  each  robot,  the  former  using  symbolic  computation  methods,  the 
latter  by  situated  action. 
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Ah  a  more  realistic  example,  consider  Luvc's  (1988)  Hystematic  examination  of  the 
nature!  of  situated  mathematics  as  a  part  of  the  Adult  Mathematics  Project.  One  of  the 
most  commonly  reported  anecdotes  from  her  work  is  the  description  of  a  man  who  is 
implementing  a  diet  program  and  trying  to  cut  his  intake  of  food  by  three-fourths,  One  of 
the  items  he  needs  to  use  in  a  recipe  is  three-fourths  of  two-thirds  of  a  cup  of  cottage  cheese. 
Instead  of  selecting  the  appropriate  school-based  algorithm  and  performing  the  necessary 
calculation  to  determine  the  appropriate  measurement,  the  man  uses  a  somewhat 
"unorthodox"  approach.  He  measures  two-thirds  cup  cf  cottage  cheese,  empties  the  contents 
»f  the  measuring  cup  on  the  counter,  shapes  the  substance  into  a  pie-shaped  pile,  cuts  it 
into  quarters,  and  removes  a  quarter  section.  The  man's  actions  are  situated,  in  the  sense 
that  he  employe  a  procedure  that  is  largely  based  on  the  context  of  the  situation,  even 
though  it  reflects  aome  school-based  knowledge  (e.  g.,  a  circle  represents  the  whole,  divide 
the  circle  into  equal  fractional  portions,  etc.).  If  there  were  many  of  these  calculations,  the 
man's  problem-solving  strategies  might  change  to  a  more  efficient  approach  that 
incorporates  some  form  of  planning  -  perhaps  he  might  even  choose  to  use  an  algorithm. 
But,  this  change  in  strategies  would  only  occur  after  the  man  stopped  his  situated  action  in 
order  to  reflect  on  the  situation  so  as  to  develop  a  more  efficient  plan  of  action. 

Schon  (1983)  has  extended  the  notion  of  situated  action  to  include  an  important 
additional  characteristic;  that  practitioners  often  reflect  on  their  situated  action  in  order  to 
deal  with  some  troublesome  phenomenon  of  the  situation.  For  Schon,  knowledge  is 
"ordinarily  tacit,  implicit  in  our  patterns  of  action  and  in  our  feel  for  the  stuff  with  which  we 
are  dealing.  It  seems  right  to  say  that  our  knowing  is  La  our  actions"  (p.  49).  As 
practitioners  become  more  experienced,  this  tacit  knowledge  increases  in  its  complexity  and 
usefulness,  but  all  practitioners  still  experience  problems  in  which  it  is  necessary  to  cope 
with  divergent  situations  where  their  situated  actions  are  not  effective  in  solving  the 
problem.  In  these  cases  practitioners  employ  reflection-in-action  to  construct  new  ways  of 
framing  the  problem  so  that  situated  action  can  resume.  Studies  of  practitioners  in  a 
variety  of  domains,  from  design  to  psychotherapy  to  town  planning,  reveal  practices  that 
alternate  between  situated  action  and  reflection-in-action.  If  such  behavior  is  characteristic 
of  practitioners,  it  may  be  advantageous  to  encourage  learners  to  adopt  similar  behavior 
when  learning  to  solve  problems. 

Assumptions  about  Knowledge 

The  nature  and  role  of  knowledge  in  the  learning  process  is  challenged  by  situated 
cognition.  Rather  than  viewing  knowledge  as  internal  to  the  nvaii,  situated  cognition 
suggests  that  knowledge  is  a  relation  between  an  individual  and  a  social  or  physical 
situation.  Greeno  (1989)  explains  that  such  a  conceptualization  is  "..analogous  to  the 
concept  of  motion  in  physics.  The  velocity  and  acceleration  of  an  object  in  motion  are  not 
properties  of  the  object  itself,  but  are  properties  of  a  relation  between  the  object  and  a  frame 
of  reference"  (p.  286).  In  fact,  the  objective  nature  of  knowledge  has  been  questioned  by 
many  in  the  social  sciences  with  some  suggesting  that  there  may  be  no  "right"  way  to 
represent  knowledge  or  structure  content  (Wilson  &  Cole,  1992).  The  view  offered  by 
Suchman  (1987)  is  that  the  common  practices  of  participants  in  social  situations  are  the 
source  of  nn  individual's  knowledge  structures  and  rules  governing  her  or  his  behavior. 
Objectivity  is  accomplished  through  "systematic  practices,  or  members'  methods  for 
rendering  our  unique  experience  and  relative  circumstances  mutually  intelligible.  The  source 
of  mutual  intelligibility  is  not  a  received  conceptual  scheme  or  a  set  of  coercive  rules  or 
norms,  but  those  common  practices  that  produce  the  typifications  of  which  schemes  and 
rules  are  made"  (pp.  57-58). 
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There  is  a  growing  body  of  evidence,  much  from  the  study  of  learning  of  mathematics, 
suggesting  that  knowledge  acquired  in  specific  situations  is  more  powerful  and  useful  than 
so-called  general  knowledge  that  is  often  decontextualized  and  represented  in  abstract 
structures  that  cannot  be  applied  in  specific  situations.  This  idea  manifests  itself  in  the 
phenomenon  that  many  people's  mathematical  constructions  are  independent  and  quite 
different  from  this  mathematics  that  people  learn  in  schools.  One  of  the  most  often  cited 
studies  in  this  rcgurd  is  the  Carraher,  Carraher,  and  Schliemann  (1985)  study  of  children's 
mathematical  abilities  in  the  streets  of  Brazil.  These  children  were  able  to  perform  complex 
mental  arithmetic  in  the  context  of  street  vending  in  their  parents'  stands.  However,  given 
pencil  and  paper  in  the  classroom,  these  same  children  were  unable  to  perform  even  the 
most  simple  calculation*,  Further,  when  the  arithmetic  problems  were  stated  in  terms 
familiar  to  the  children,  they  were  still  unable  to  perform  the  calculations.  Even  though  the 
study  may  bo  somewhat  flawed  due  to  the  mixing  of  ethnographic  methods  with 
quantitative  analyses,  thu  results  remain  compelling. 

Saxe  (1989)  HUggestud  that  these  children  had  certain  limits  in  their  ability  to  perform 
the  school-based  tasks,  specifically  that  the  children  in  the  Carraher,  et  al.,  study  had  little 
or  no  school-baaed  mathematical  knowledge  (orthographic  mathematics).  Saxe  argued  that 
the  differences  would  disappear  if  the  children  had  more  prior  knowledge  of  school-based 
learning.  Working  with  the  Carrahers  in  Brazil,  Saxe  carried  out  a  similar  study  using 
children  who  were  street  sellers,  urban  children  from  the  same  community  who  were  not 
involved  in  street  selling,  and  a  group  of  rural  children  with  limited  exposure  to  currency  and 
transactions  with  currency.  The  latter  two  groups  were  matched  with  the  sellers  on  age  and 
school  background.  Performance  on  the  orthographic  problems  was  clearly  related  to 
educational  background.  Those  students  with  less  than  a  third  grade  level  of  education 
correctly  solved  40%  of  the  problems,  while  children  with  more  than  a  fourth  grade  level 
solved  close  to  90%  of  the  problems  correctly.  In  analyzing  strategies,  the  children  with  more 
schooling  tended  to  make  use  of  school-based  algorithms  in  the  context  of  the  street 
calculations.  Apparently,  it  is  not  that  cognition  is  situated,  but  that  the  greater  the 
knowledge  of  school-based  mathematics,  the  more  likely  a  person  will  use  school  bun  eel 
knowledge  in  other  contexts. 

Does  the  knowledge  gained  in  the  street  transfer  to  the  school?  In  the  Saxe  (1HH0) 
study,  second  and  third  grade  sellers*  and  non-sellers1  performance  on  school-like 
computations  and  word  problems  revealed  that  second-grade  sellers  performed  better  than 
thu  second-grade  non-sellers.  This  result  was  attributed  to  the  fact  that  the  second-grade 
sellers  used  strategies  that  were  very  much  like  those  used  in  selling  candy  in  the  street,  By 
the  third  grade,  the  non-sellers  caught  up  to  the  sellers;  however,  the  sellers  still  used  many 
of  the  strategies  they  learned  in  the  street,  while  the  non-sellers  continued  to  use  their 
school-based  knowledge.  The  results  suggest  that  the  extent  to  which  mathematical 
knowledge  is  situated  is  much  less  than  that  suggested  by  the  Carraher,  et  al.  study. 
Apparently  the  better  a  child  can  use  school-based  knowledge,  the  more  it  can  be  used  in 
other  situations,  and  knowledge  that  children  acquire  outside  of  school  can  be  used  in  school 
to  some  extent.  The  degree  to  which  this  knowledge  is  ultimately  useful,  however,  will 
certainly  be  dependent  upon  the  contexts  for  learning  provided  in  the  schools. 

Shirley  Brice  Heath  (1983)  conducted  an  extensive  ethnography  in  Appalachia  that 
focused  upon  three  different  communities;  its  purpose  was  to  better  understand  how 
learning  was  valued  in  the  institutions  within  these  communities,  After  determining  the 
kind  of  knowledge  and  abilities  that  were  valued  within  the  three  communities,  school- 
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based  programs  were  designed  to  take  advantage  of  cognitive  abilities  and  remediate 
cognitive  weaknesses.  The  instructional  intervention  wore  quite  successful,  focusing  on  the 
knowledge  utilized  in  the  community  domain  and  how  the  knowledge  within  the  school 
domain  could  be  made  to  correspond  to  the  community  knowledge,  In  order  to  help  learners 
to  find  these  correspondences,  a  variety  of  translation  processes  were  used.  For  example, 
community  verbal  knowledge  largely  consisted  of  opinions,  sayings,  proverbs,  and  the  like. 
The  corresponding  verbal  knowledge  in  school  was  predominantly  written,  consisting  of 
facts,  scientific  principles,  scientific  methods  and  the  like,  Translations  would  require  the 
specification  of  gaps  between  knowledge  types  and  identification  of  common  elements.  This 
explicit  activity  helped  learners  see  the  utility  of  both  forms  of  knowledge.  Furthermore, 
many  of  these  activities  brought  the  community  into  the  classroom,  meaning  that  many 
family  and  community  members  participated  more  fully  in  the  education  of  the  children. 

Clearly,  this  method  has  broader  social  aspects.  The  increased  involvement  of 
significant  others  in  the  schooling  process  increases  the  motivation  of  the  learners,  helping 
them  to  see  the  importance  of  schooling,  If  it  is  important  for  a  parent  to  come  to  class,  then 
the  information  that  is  being  learned  there  perhaps  has  some  merit.  Given  the  above 
evidence,  it  becomes  apparent  that  knowledge  is  acquired  in  both  formal  and  informal  ways, 
and  that  these  types  of  knowledge  can  interact  through  purposeful  activities  that  are 
designed  to  take  advantage  of  the  strengths  of  one  or  the  other  knowledge  types.  It  may 
also  be  concluded  that  all  knowledge  is  not  necessarily  situated,  as  advocated  in  the  radical 
view  of  situated  cognition,  Formal  knowledge  can  he  used  in  an  informal  context  if  the 
formal  knowledge  is  understood  well  enough.  The  question  is,  is  school-based  knowledge 
distinct  from  situated  knowledge?  The  evidence  suggests  that  there  is  overlap  and 
interaction, 


Characteristics  of  Situated  Learning 

Learning  requires  more  than  just  thought  and  action,  or  a  particular  physical  or  social 
situation,  or  just  receiving  a  body  of  factual  knowledge;  it  also  requires  participation  in  the 
actual  practices  of  the  culture,  Adopting  a  relational  view  of  knowledge  and  situated  activity 
changes  the  focus  of  the  analysis  of  learning  to  characteristics  of  social  participation  in 
communities  of  practice.  Lave  and  Wenger  (H)IH)  suggest  that  participation  is  a  key 
clement,  requiring  negotiation  of  meanings  in  various  situations,  with  the  result  that 
"understanding  and  experience  are  in  constant  interaction"  (p.  51-52).  As  discussed  below, 
a  variety  of  approaches  to  the  design  of  environments  that  promote  the  processes  of  situated 
learning  have  been  developed. 

Cognitive  Apprenticeship  or  Legitimate  Peripheral  Participation? 

One  of  the  most  widely  discussed  features  of  situated  cognition  as  it  was  originally 
proposed  is  the  notion  of  cognitive*  apprenticeships  as  a  means  for  learners  to  become 
participants  in  communities  of  practice,  As  described  by  Brown  and  his  colleagues  (Brown, 
Collins  &  Duguid,  19H9;  Collins,  Brown  &  Newman,  1989),  cognitive  apprenticeships 
provide  a  general  framework  with  four  components  -  content,  methods,  sequence,  and 
sociology  -  for  designing  learning  environments.  Content  includes  domain  knowledge, 
heuristic  strategies,  control  strategies,  and  learning  strategies,  all  of  which  are  well 
explicated  in  the  literature  on  cognitive  learning  theories  and  instructional  design.  Methods 
refer  to  teaching  methods,  including  modeling,  coaching,  scaffolding,  fading,  articulation, 


reflection,  and  exploration.  Instructional  sequence  includes  notions  such  as  increasing 
complexity,  increasing  diversity,  and  global  before  local  skills.  Sociology,  the  only  category 
that  seems  to  have  emerged  directly  from  theories  of  situated  cognition,  includes  five 
aspects:  situated  learning,  culture  of  expert  practice,  intrinsic  motivation,  exploiting 
cooperation,  and  exploiting  competition.  Motivation,  cooperative  learning  and  competition 
have  each  played  a  role  in  the  design  and  development  of  instructional  systems  for  some 
time.  That  leaves  only  the  first  two  as  something  to  be  incorporated  into  our  design  models. 
The  second,  the  culture  of  expert  practice  is  actually  something  that  should  always  be 
examined  in  instructional  design.  Part  of  the  analysis  phase  of  design  is  to  determine  the 
tasks  to  be  performed  and  how  they  are  performed,  induing  the  devices  that  are  employed 
and  the  skills  required  to  use  such  devices.  The  only  difference  is  that  Collins,  et  al.  argue 
for  a  fairly  "rich"  analysis  that  incorporates  many  subtle  aspects  of  the  culture,  including  all 
of  the  various  nuances  of  communication  that  occur  in  expert  practice.  Rather  than  taking 
the  final  aspect,  situated  learning,  to  mean  that  the  knowledge  ought  to  be  situated  within 
the  context  in  which  it  exists,  Collins,  et  al.  imply  that  knowledge  can  be  situated  if  it  is 
acquired  in  the  process  of  learning-by-doing.  If  learning  involves  some  activity  encompassing 
the  to-be-learned  knowledge  that  requires  the  active  participation  of  the  learner,  then 
according  to  Collins,  et  al.,  this  knowledge  is  situated. 

It  is  curious  that  the  only  element  of  the  Collins,  et  al.  model  that  directly  relates  to 
situated  cognition  is  a  distinct  component.  Rather  than  trying  to  integrate  situated 
cognition  into  a  model,  it  is  left  as  a  separate  component.  The  use  of  the  term 
apprenticeship  has  also  been  criticized  because  the  characteristics  of  master-apprentice 
relationships  in  many  cultures  may  represent  too  narrow  a  view  of  situated  learning.  Lave 
and  Winger  (1991)  prefer  to  describe  the  central  concept  of  situated  learning  as  legitimate 
peripheral  participation,  in  order  to  promote  a  decentered  view  of  learning  that  shifts 
analysis  away  from  the  notion  of  "master  as  locus  of  authority"  to  an  analysis  of  "the 
intricate  structuring  of  a  community's  learning  resources"  (p.  94).  The  notion  of  legitimate 
peripheral  participation  differs  from  cognitive  apprenticeships  in  several  ways,  not  the  least, 
of  which  is  that  the  terminology  does  not  include  the  social  connotations  commonly 
associated  with  the  word  apprenticeship.  For  Lave  and  Wenger,  learners  must  be 
"legitimate  members  of  the  community,  not  passive  observers,  and  their  activities  must  be 
performed  in  the  context  of  the  work  of  the  community.  "Peripheral"  participation  refers  to 
the  fact  that  by  their  nature  as  novices,  learners  cannot  be  full  participants  in  all  community 
activities,  hut  at  the  same  time  they  must  be  recognized  as  participants  in  some  aspects  of 
the  work  of  the  community.  There  should  be  time  to  learn,  and  to  discuHs  ideas  with  peers 
and  old-timers.  The  old-timers  should  not  be  threatened  by  the  potential  of  the  newcomer, 
but  should  be  in  a  position  to  offer  the  best  of  their  knowledge  and  skill.  "Participation" 
means  apprentices  (newcomers)  should  be  doing  the  thing  that  they  are  learning  to  do,  not 
just  observing. 

Lave  and  Wenger  suggest  that  legitimate  peripheral  participation  is  the  mechanism  of 
enmlturation  for  a  learner  that  includes  relationships  between  apprentices  and  masters, 
but  also  includes  all  of  the  other  participants,  skills,  artifacts,  symbols,  and  ideas  that  are 
part  of  the  culture  of  practice.  As.  a  result,  their  study  of  legitimate  peripheral  participation 
focuses  on  that  form  of  social  participation  which  includes  learning  as  a  necessary 
component.  Learning  as  a  characteristic  of  social  interaction  cannot  be  extricated  from  its 
legitimate  context.  Unfortunately,  legitimate  peripheral  participation  is  not  an  instructional 
method.  Rather,  it  is  a  lens  for  viewing  and  understanding  learning  in  new  ways. 
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Patterns  of  Situated  Learning 


Legitimate  peripheral  participation  serves  to  help  learners  develop  a  holistic  view  of 
what  the  community  of  practice  is  about,  and  what  there  is  to  learn  within  that  community. 
Opportunities  for  learning  are  structured  by  the  requirements  of  work,  rather  than  teacher- 
student  relations.  In  fact,  Lave  and  Wenger  (1991)  note  that  in  many  situations 
apprentices  learn  mostly  from  other  apprentices.  For  example,  among  Liberian  tailors 
learning  their  trade  (Lave,  in  preparation,  as  cited  in  Lave  &  Wenger,  1991),  much  of  the 
communication  of  technique  comes  from  more  experienced  peers.  The  old-timer  (master) 
mainly  serves  as  a  model  for  the  ideal  professional  in  the  field  performing  the  daily  duties, 
and  only  to  a  lesser  degree  does  the  master  provide  formal  instruction.  What  they  also 
learn  is  how  to  talk  within  the  community.  The  shared  practice  within  a  community 
includes  both  "talking  within"  to  share  information  about  ongoing  activities,  and  "talking 
about"  through  stories  that  support  "communal  forms  of  memory  and  reflection"  (p.  109). 
The  case  of  the  "nondrinking"  alcoholics  (Cain,  in  preparation,  as  cited  in  Lave  &  Wenger, 
1991)  demonstrates  that  th~  ?le  of  language  in  situated  learning  does  not  just  contribute  to 
the  development  of  complex  skills.  Predominantly,  the  learning  activities  employed  in  the 
Alcoholics  Anonymous  organization  are  based  upon  language,  in  the  form  of  public  speaking 
to  groups  and  writing  in  newsletters.  Older  members  adapt  their  personal  story  tellug  to 
the  experiences  of  newcomers,  while  newcomers  learn  to  construct  personal  stories  from  the 
models  that  are  provided. 

Lave  and  Wenger  (1991)  also  suggest  that  learning  in  communities  of  practice  is  nnt 
highly  structured  and  sequenced,  but  rather  "unfolds  in  opportunities  for  engagement  in 
practice"  (p.  93).  There  are  no  rules  dictating  what  should  be  learned,  or  when  it  should  he 
learned.  Opportunities  to  learn  are  mainly  improvised  from  the  situation  at  hand,  following 
a  curriculum  that  includes  the  resources  of  everyday  practice,  Mayan  midwiveH  (Jordan, 
1989,  as  cited  in  Lave  &  Wenger,  1991)  do  not  provide  any  explicit  instruction  ut  all.  In 
fact,  the  midwives  report  that  much  of  their  knowledge  comes  to  them  in  dreams.  The  data 
indicates  that  they  actually  learn  their  profession  by  observing  the  practices  and 
participating  in  increasingly  complex  practices  from  a  very  young  age.  U.S.  Navy 
quartermasters  (Hutchins,  in  press,  as  cited  in  Lave  &  Wenger,  1991)  stated  that  they 
preferred  apprentices  (newcomers)  who  had  not  received  the  traditional  classroom 
instruction  prior  to  coming  on-board,  because  it  took  more  time  to  correct  this  erroneous 
learning  than  it  did  to  work  with  someone  who  knew  little  or  nothing.  Also,  while  there  was 
some  "traditional"  instructional  materials  (workbooks)  on  hoard,  most  of  the  learning 
occurred  in  the  context  of  the  actual  operation  of  the  ship.  Newcomers  were  to  perform 
inc.*  -asingly  more  complex  tasks  in  a  spiral  curriculum  that  included  previously  learned 
tasks.  These  tasks  were  performed  under  the  constant  guidance  of  the  old-timer,  who  would 
correct  errors  or  take  over  the  task  if  needed.  The  case  of  the  Liberian  tailors  (Lave,  in 
preparation,  as  cited  in  Lave  &  Wenger,  1991)  again  confirms  that  curriculum  can  emerge 
from  the  practice  of  formal  apprenticeships.  The  situations  involving  the  Mayan  midwives 
and  the  Navy  quartermasters  were  not  strictly  defined  as  apprenticeships,  but  the  Liberian 
tailors  entered  into  a  formal  contract  with  the  old-timer.  An  interesting  attribute  of  the 
tailors'  learning  is  that  the  curriculum  is  reversed.  Rather  than  beginning  with  the  initial 
steps  in  the  procedure  of  making  clothes,  the  apprentices  begin  at  the  end,  working  on 
finishing  touches  first  in  order  to  observe  the  quality  of  workmanship  and  models  for  the 
finished  product.  Ip  all  of  the  cases  cited  here,  curriculum  is  situated  in  practice?,  and  cannot 
he  "considered  in  isolation,  manipulated  in  arbitrary  didactic  terms,  or  analyzed  apart  from 
the  social  relations  that  shape  legitimate  peripheral  participation"  (p.  98). 


Structuring  Learning  (Instruction?)  from  a  Situated  View 


As  mentioned  above,  the  descriptions  of  learning  provided  by  theories  of  situated 
cognition  are  based  upon  assumptions  about  learning  that  are  vastly  different  from  those 
embodied  in  current  instructional  design  models.  Several  general  principles  have  been 
suggested  that  can  guide  instructional  designers  in  the  development  of  strategies  to 
facilitate  situated  learning.  For  example,  the  Cognition  and  Technology  Group  at  Vanderbilt 
(1992)  suggests  that  "students  need  to  engage  in  argumentation  and  reflection  as  they  try  to 
use  and  then  refine  their  existing  knowledge  and  attempt  to  make  sense  of  alternate  points 
of  view"  (p.  67).  They  emphasize  that  instruction  should  be  "anchored"  in  meaningful 
contexts  that  allow  si'uated  learning  to  be  simulated  in  classrooms  (Cognition  and 
Technology  at  Vanderbilt,  1991).  In  this  way,  environments  can  be  designed  that  allow 
"sustained  exploration"  of  the  various  aspects  of  a  problem,  helping  students  to  "understand 
the  kinds  of  problems  and  opportunities  that  experts  in  various  areas  encounter  and  the 
knowledge  that  these  experts  use  as  tools"  (Cognition  and  Technology  Group  at  Vanderbilt, 
1992,  p.  67).  This  sustained  exploration  can  also  be  facilitated  by  technology,  as  in  the 
approach  to  developing  cognitive  flexibility  suggested  by  Spiro  and  his  colleagues  (Spiro, 
Feltovich,  Jacobson,  &  Coulson,  1991;  Spiro  &  Jehng,  1991).  In  this  application,  learners 
examine  various  cases  or  scenarios  from  many  perspectives,  assisted  by  the  rapid  and 
efficient  access  to  information  provided  by  hypermedia  technology.  Rieber  (1992)  echoes 
these  suggestions  in  discussing  guidelines  for  designing  computer-based  microworlds.  He 
advocates  the  design  of  meaningful  contexts  that  support  self- regulated  learning, 
establishing  a  spiral  curriculum,  and  nurturing  incidental  learning. 

As  noted  above,  one  area  that  will  require  extensive  consideration  in  designing 
environments  based  on  principles  of  situated  learning  is  the  amount  of  control  that  is 
provided  to  the  learners.  Current  models  of  instructional  design  tend  to  assume  a  great 
degree  of  control  for  the  teacher  (or  the  "system")  with  respect  to  sequencing,  strategies, 
questioning,  etc.  Tobin  and  Dawson  (1992)  note  that  this  problem  leads  to  a  "dysfunctional 
learning  environment"  in  which  learners  have  "little  autonomy,  and  hence  lose  interest  in  the 
curriculum"  (p.  91).  Several  alternative  approaches  to  teaching  have  been  cited  as  examples 
that  hotter  support  situated  learning.  One  example  is  Schoenfeld's  (1985)  approach  to 
mathemutics  instruction,  in  which  the  teacher  explicitly  models  problem-solving  strategies  to 
the  students,  and  students  are  given  chances  to  generate  their  own  problems.  Reciprocal 
teaching  (Palinscar  &  Brown,  1984)  is  another  approach  in  which  control  of  the  learning 
activities  is  given  to  the  learners.  In  this  method,  small  groups  of  learners  assume  roles 
including  "teacher",  "critic",  and  "producer"  in  the  process  of  comprehending  written 
passages.  Control  of  learning  activities  and  communication  among  learners  also  can  be 
encouraged  with  technology,  as  in  the  Computer-Supported  Intentional  Learning 
Environment  (CSILE  project  described  by  Scardmalia  and  Bereiter,  1991).  At  a  school 
where  CSILE  has  been  tested,  children  of  varying  grade  levels  share  ideas,  criticisms  and 
explanations  over  a  computer  network,  reflecting  characteristics  of  the  legitimate  peripheral 
participation  discussed  earlier.  In  these  and  many  other  similar  cases,  the  role  of  the 
teacher  seems  to  be  more  like  "Yoda"  from  the  "Star  Wars"  trilogy,  rather  than  "Professor 
Kingsfield"  as  depicted  in  the  television  series  "Paper  Chase".  Learners  are  in  control,  while 
the  teacher  serves  as  a  model  and  facilitator  rather  than  directly  controlling  the  learning 
process. 
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Conclusions 


After  careful  consideration  of  the  duta  rited  in  support  of  situated  li!iirniiiKi  it  is 
evident  that  integrating  the  principles  of  learning  suggested  tiy  the  proponents  of  the  theory 
into  current  instructional  design  practice  will  not  he  cany.  In  fart,  wit  agree  with  Hrown  and 
Duguid  (1993)  that  what  is  necessary  is  a  complete  redefinition  of  tiiHtmctton  and  teaching. 
There  are  just  too  many  problems  involved  in  trying  to  make  explicit  the  knowledge  of 
experts,  to  abstract  that  knowledge,  and  to  communicate  the  knowledge  to  novice  learners. 
In  the  end, we  are  left  With  the  question  of  whether  there  can  be  an  instructional  theory 
based  on  situated  cognition,  and  what  the  role  of  technology  might  be  in  such  a  theory.  It  is 
apparent  that  we  need  to  focus  efforts  on  the  development  of  tools  and  environments  that 
support  communication  and  collaboration  among  learners  and  experts,  as  has  been 
successfully  demonstrated  in  the  examples  cited  above.  Beyond  that,  perhaps  Tripp  (1993) 
is  correct:  we  should  build  and  test  the  artifacts  before  we  formulate  the  theory. 
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The  unprecedented  freedom  for  users  to  control  the  ncope  and  sequence  of  their 
interactions  with  hypermedia  systems  presents  many  challenges  to  those  who  design  and 
study  these  systems  in  educational  settings.  Early  efforts  to  develop  hypermedia  systems 
revealed  that  the  inherent  node-link  structure  is  both  advantageous  and  problematic  (see 
Coiiklin,  19H7).  Rut  when  users  have  the  freedom  i >  follow  any  of  a  multitude  of  link 
permutations,  disorientation  often  results,  Further,  without  appropriate  training,  novice 
users  do  not  possess  the  strategies  necessary  for  effective  "browsing"  of  large  hypermedia 
documents  (Duffy  &  Knuth,  1991),  Many  designers,  therefore,  advocate  that  features  such 
as  visual  maps,  search  facilities,  and  guided  tours  be  included  in  hypermedia  systems  to 
alleviate!  some  of  these  problems  (e.g.  Hammond,  1989;  Laurel,  1990,  1991). 

With  the  (•mergence  of  hypermedia  systems  as  a  major  architecture  for  educational 
and  other  information -oriented  software  comes  the  related  problem  of  how  to  document  and 
analyze  user  interactions  with  these  systems  for  the  purposes  of  research  and  evaluation. 
Many  hypermedia  systems  to  date  have  employed  a  "browsing"  interface,  but  alternative 
approaches  are  also  emerging  (See  Nelson  &  Palumbo,  1992).  Regardless  of  the  type  of 
interface,  many  questions  can  be  generated  when  studying  the  interactions  of  users  with 
hypermedia  systems.  For  example,  how  many  users  chose  to  follow  a  particular  link,  and 
why  was  one  link  chosen  over  another?  How  does  the  choice  of  one  link  affect  choices  of 
subsequent  links?  When  are  graphic  images,  animations,  and  video  segments  accessed? 
What  user  tasks  are  appropriate  for  guiding  interaction  with  the  system?  Are  there 
patterns  of  navigation  that  lead  to  more  complete  learning?  These  and  many  other 
questions  need  answers  when  designing,  developing,  anJ  evaluating  hypermedia 
applications  for  education  and  other  settings. 

A  wealth  of  user  interaction  data  can  be  easily  collected  within  many  hypermedia 
development  environments  in  order  to  study  aspects  of  the  interface,  including  the  nature  of 
user  navigation  patterns,  the  time  spent  at  each  node,  and  the  use  of  help  and  other 
orienting  facilities.  The  data  can  represent  the  paths  a  user  follows  through  the  system, 
and  the  choices  made  at  each  node  in  the  system.  The  problem  is  that  because  of  the 
nature  of  this  data,  traditional  methods  of  analysis  such  as  surveys  or  pretest-posttest 
designs,  are  not  particularly  effective  for  determining  usability  or  comparing  alternative 
interface  designs.  Researchers  have  had  to  develop  new  techniques  for  analyzing  patterns  of 
user  interaction  in  order  to  evaluate  the  design  and  effectiveness  of  hypermedia  systems  (e. 
g.  Misanchuk  &  Schwier,  1992).  There  is  a  need  to  categorize  and  compare  groups  of  users 
in  order  to  compare  the  effectiveness  of  alternative  system  features,  as  well  as  describing 
characteristics  of  interaction  by  individual  users  within  the  same  system.  This  paper  will 
discuss  several  methods  for  visualizing  and  characterizing  user  interactions  with 
hypermedia  systems  that  have  been  found  effective  in  several  development  and  evaluation 
studies. 

Characterizing  user  interactions  using  path  algebras 

Characterizing  the  interactions  of  individual  users,  or  comparing  groups  of  users,  can 
be  accomplished  using  several  methods  derived  from  mathematical  set  and  graph  theories 
(Backhouse  &  Carre,  1975;  Carre,  1971).  In  the  first  method,  path  algebras  are  employed 
to  describe  and  compare  the  routes  users  take  through  hypermedia  systems  (Alty,  1984), 
Users  may  follow  a  variety  of  types  of  paths  through  the  two-dimensional  space  of  a 
hypermedia  database  (Canter,  Rivers  &  Storrs,  1985).  Using  simple  programming 
techniques,  it  is  possible  to  "trap"  and  save  in  data  files  the  sequence  of  nodes  visited  by  the 
user,  along  with  the  more  specific  user  actions  taken  at  individual  nodes  such  as  clicks  on 


buttons,  menu  selections,  or  viewing  graphic  images,  animations,  or  audio  and  video 
segments. 

The  data  files  can  be  analyzed  using  algorithms  that  extract  the  nature  of  the 
"subpaths"  of  the  total  path  followed  by  the  user.  The  types  of  subpaths  users  follow 
through  hypermedia  systems  can  include  rings,  loops,  paths,  and  spikes  (See  Figure  1).  As 
described  by  Canter,  Rivers,  and  Storrs  (1985),  paths  are  movements  between  nodes  in  a 
linear  sequence  from  one  node  to  another.  Spikes  occur  when  a  user  moves  through  a 
number  of  nodes,  and  then  returns  along  the  path  to  the  initial  node.  Loops  and  rings  are 
similar  in  that  the  user  moves  through  a  set  of  nodes  that  eventually  returns  to  the  original 
node  in  a  circular  pattern.  Rings  are  created  when  a  user  loops  through  several  patterns 
that  visit  nodes  that  were  previously  accessed.  The  path  that  an  individual  user  follows 
while  interacting  with  a  hypermedia  system  gives  an  indication  of  the  nature  of  that 
interaction  at  different  points  in  time.  Of  course,  the  types  of  paths  a  user  might  follow  will 
be  determined  in  part  by  the  organizational  structure  of  the  system.  Hypermedia  systems 
organized  hierarchically  will  tend  to  generate  paths  with  many  spikes,  as  the  user  moves 
down  a  "branch*'  of  the  tree  and  then  hack  to  the  "root".  A  linear  organization  for  the 
hypermedia  system  will  tend  to  product!  many  paths.  Rings  or  loops  tend  to  indicate  that  a 
user  is  revisiting  nodes,  perhaps  to  search  tor  information,  or  perhaps  because  of 
disorientation. 

Figure  1.  Types  of  user  paths  through  hypermedia  nyNtoms. 
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The  frequencies  of  each  type  of  path  followed  during  an  individual  user's  interaction 
provides  a  basis  for  comparison  between  individuals  and  groups.  These  frequencies  can  be 
recorded  for  each  user,  and  common  parametric  statistical  procedures  can  be  employed  to 
compare  the  mean  number  of  each  path  type  for  groups  of  users  in  order  to  determine 
whether  the  frequencies  were  significantly  different.  This  approach  provides  a  means  for 
analyzing  individual  users1  interactions  that  can  be  more  precise  than  merely  determining 
the?  frequency  that  one  node  is  accessed  from  another  node  (Misanchuk  &  Schwier,  1992). 
There  is  also  the  possibility  of  comparing  groups  of  users  that  is  not  afforded  by  other 
mothods  that  focus  on  individual  user  access  patterns.  In  this  way,  alternative  interface 
design  strategies  can  be  compared  using  quantitative  methods  that  may  reveal  several 
advantage*  of  one  riomgn  over  another. 
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This  method  was  employed  with  data  collected  in  a  prior  study  (Nelson,  1991)  to 
further  characterize  the  interaction  patterns  of  various  users  who  possessed  different  levels 
of  prior  knowledge  for  the  domain  presented  in  a  hypermedia  system.  Two  groups  of  users 
with  high  and  low  prior  knowledge  of  the  content  completed  one  of  three  tasks  with  a 
hypermedia  document  (browse,  search,  or  study).  The  results  indicated  that  prior 
knowledge  influenced  interaction  patterns  (e.g.,  there  were  more  "spikes"  for  individuals 
with  lower  prior  knowledge).  In  addition,  the  results  of  this  analysis  also  confirmed  a  design 
assumption  that  was  made,  namely,  that  definitions  and  examples  of  unfamiliar  terms 
needed  to  be  provided  to  users  with  low  prior  knowledge,  but  that  these  definitions  and 
examples  would  not  be  accessed  by  users  with  high  prior  knowledge.  The  large  number  of 
"spikes"  followed  by  some  users  was  the  result  of  their  access  of  definition  and  example 
nodes  for  some  concepts  in  the  document.  It  appears  that  this  method  of  analysis  of  user 
interactions  with  hypermedia  systems  can  be  useful  in  testing  design  alternatives. 
Extensions  of  this  technique  might  utilize  similar  algorithms  embedded  within  the 
hypermedia  system  to  detect  in  real  time  some  of  the  various  interaction  patterns,  and  to 
provide  system  interventions  or  guidance  based  on  such  analyses. 

Characterizing  user  interactions  using  directed  graphs 

The  second  method  employs  graph  theory  to  construct  a  network  representation  of  the 
paths  taken  by  individual  users  through  a  hypermedia  document,  and  to  compare  the 
networks  as  generated  by  the  Pathfinder  algorithm  (Schvaneveldt  et  al.,  1985).  Pathfinder 
was  originally  developed  to  study  the  associative  characteristics  of  human  memory,  and  has 
been  applied  in  a  variety  of  research  studies  (Schvaneveldt,  1990).  The  algorithm  is 
applied  to  proximity  data  that  represents  participants1  ratings  of  the  degree  of  relatedness 
between  a  set  of  concepts.  The  resulting  proximity  matrices  are  analyzed  by  Pathfinder  to 
construct  a  network  representation.  Correlations  can  then  be  computed  between 
individuals1  networks  to  determine  the  degree  to  which  the  networks  are  similar. 

When  used  to  study  navigation  patterns  in  hypermedia,  this  method  assumes  that  the 
more  frequently  a  link  is  traversed  by  a  user,  the  more  direct  is  the  association  in  the 
resulting  Pathfinder  network  that  represents  the  interaction  pattern.  To  analyze  the  data, 
proximity  matrices  are  constructed  to  represent  the  relative  "proximity"  of  a  node  to  every 
other  node  in  the  hypermedia  document.  In  this  case,  proximity  is  defined  as  the  number  of 
times  a  node  was  visited  through  a  link  from  another  node.  As  shown  in  Figure  2,  the  raw 
data  is  used  to  construct  the  proximity  matrix  by  analyzing  the  user's  sequential  moves 
between  nodes.  Each  move  from  one  node  to  another  results  in  an  increase  in  the  frequency 
value  of  the  corresponding  cell  in  the  matrix.  So  a  move  from  node  1  to  node  7  would 
increase  the  matrix  cell  at  row  1 -column  7  by  one,  and  a  move  from  node  7  to  node  3  would 
increase  the  matrix  cell  at  row  7-column  3  by  one,  and  so  on  through  the  sequence  of  the  raw 
data.  After  individual  matrices  have  been  developed  for  each  user,  group  data  can  be 
derived  by  calculating  the  average  value  for  each  cell  in  the  matrix  (See  Figure  2). 


632 

CO 


Figure  2.  Deriving  a  proximity  matrix  from  raw  data,  and  averaging  across  a  group  of  users. 
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Once  the  proximity  matrices  are  established,  the  Pathfinder  algorithm  can  extract  the 
representative  network  from  the  average  matrix,  as  shown  in  Figure  3.  A  computer-based 
tool  for  analyzing  data  using  the  Pathfinder  algorithm  is  available  (Interlink,  1990).  The 
relatedness  coefficients  used  to  plot  various  Pathfinder  solutions  from  different  data  sett  can 
he  correlated  to  determine  the  degree  of  similarity  of  the  paths  taken  by  individuals  or 
groups.  The  correlation  values  indicate  the  degree  to  which  different  groups  of  users  followed 
similar  paths  through  the  hypermedia  document.  This  method  was  also  applied  to  the  data 
from  the  study  cited  above.  Correlations  between  the  groups  suggested  that  characteristics 
of  the  task  may  have  influenced  the  interaction  patterns  of  users.  At  least  for  this  data  set, 
searching  and  browsing  tasks  produced  patterns  that  were  more  similar  than  for  the  study 
task,  as  indicated  in  the  comparatively  high  correlations.  This  method  may  also  be 
extended  by  hypermedia  authors  to  specify  the  initial  organization  of  the  nodes  and  links  in 
a  hypermedia  document  (McDonald,  Paap  &  McDonald,  1990). 

A  Picture  of  the  Users' Interaction 

Another  approach  to  visualizing  the  interaction  path  of  an  individual  user  is  to  construct 
an  image  based  on  the  interaction  data.  The  interaction  that  we  wanted  to  represent  was  the 
amount  of  time  spent  on  each  screen,  the  direction  of  links,  and  the  number  of  times  that  a 
screen  was  visited.  In  order  to  do  this,  each  screen  of  information  that  was  visited  by  the  user  is 
represented  as  a  rectangle.  The  more  time  that  is  spent  on  a  node,  the  larger  the  rectangle 
becomes.  When  a  learner  links  from  one  piece  of  information  to  another,  a  line  segment  is 
drawn  with  an  endpoint  on  the  originating  node  and  an  arrowhead  on  the  destination  node. 


9 

ERIC 


una 


Further,  the  more  times  a  learner  visits  a  node,  the  darker  it  WocomeN  with  a  range  of  4  (white, 
light  gray,  dark  gray,  hlack).  The  resulting  image  would  then  represent  the  interaction  of  that 
user. 


Nelson's  (1991)  data  were  used  in  the  Toolbook  program  to  generate  an  image  of  the 
interaction  each  student  had  with  the  system.  There  were  a  total  of  57  images  produced  (one 
for  each  subject).  In  the  Nelson  study,  each  subject  was  a  student  in  an  undergraduate 
educational  psychology  class.  The  system  that  they  examined  was  a  hypermedia  program  on 
the  field  of  educational  psychology.  There  were  three  groups,  The  Study  group  was  told  that 
they  will  have  a  test  on  the  content  in  the  program.  The  Browse  group  was  told  to  evaluate  the 
program  because  it  might  be  used  in  later  classes,  The  Index  group  was  asked  to  find 
definitions  to'  a  list  of  words. 

Figure  3.  Pathfinder  solution  for  hypothetical  data. 


The  57  data  files  for  the  subjects  in  the  Nelson  (1991)  study  were  entered  into  the 
Toolbook  program  to  generate  a  map  of  each  individual  learners  interaction  with  the 
program.  These  57  images  were  printed  and  copies  were  made.  The  images  were  given  to 
three  different  groups:  Gifted  middle  school  students  (n=6),  teachers  in  an  introductory 
educational  computing  course  (n=6),  and  instructional  technology  doctoral  students  (n=6). 
The  first  two  groups  were  given  the  57  images  and  some  paper  clips  and  were  instructed  to 
place  the  images  on  a  table  and  try  to  categorize  the  images  into  3  to  5  groups.  They  were 
then  asked  to  write  a  description  of  the  procedure  they  followed  to  create  the  categories  and 
to  describe  the  categories.  The  third  group  did  the  same  thing,  except  they  were  told  what 
the  images  represented  and  each  image  indicated  which  group  from  the  Nelson  study  the 
image  was  from. 

The  only  performance  data  available  from  the  Nelson  study  was  the  course  grade 
each  subject  received.  While  this  is  a  very  indirect  measure  of  the  impact  of  the  instructional 
package,  it  was  the  only  one  available.  Grades  in  the  sample  were  either  A,  B  or  C, 
Unfortunately,  for  a  variety  of  reasons,  only  38  of  the  57  images  could  be  associated  with  a 
grade.  The  categories  generated  by  the  groups  in  the  current  study  were  then  used  in  a  chi- 
squared  analysis.  One  dimension  was  the  grade  received  by  the  learner  a' id  the  other 
dimension  was  the  category  generated  by  the  subjects  in  this  study.  A  to«,al  of  18  chi- 
squared  tests  were  conducted. 
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Results 

Results  indicated  that  there  was  only  one  subject  whose  categorization  strategy 
resulted  in  a  statistically  significant  result  in  regards  to  the  grade  received  in  the  course(»eot 
Figure  4).  The  categories  were:  (W)  "Has  more  dark  squares  than  any  other  kind"  (see, 
Figure  5);  (X)  "Has  more  white  squares  than  any  other  kind"  (see,  Figure  6);  (Y)  "Has  more 
polka  dot  (light  gray)  squares  than  any  other  kind;"  and,  (Z)  "Two  highest  patterns  were 
tied."  Unfortunately,  expected  values  are  very  low  in  the  lower  categories  and  would  result 
in  questionable  conclusions.  However,  given  the  exploratory  nature  of  this  research 
methodology,  it  is  worth  taking  a  closer  look. 

Figure  4,  Summary  table  and  observed  frequencies  for  the  only  significant  categorization 
scheme  based  on  grade  in  the  course  and  created  by  N2. 


Summary  Table  for  Grade,  N2  Observed  Frequencies  tor  Grade,  N2 


Num.  Missing 

0 

W 

X 

Y 

Z 

Totals 

CF 

6 

A 

12 

9 

0 

2 

23 

Chi  Square 

19.886 

B 

5 

7 

0 

1 

13 

Chi  Square  P-Value 

.0029 

C 

0 

1 

1 

0 

2 

G-Squared 

• 

Totals 

17 

17 

1 

3 

38 

G-Squared  P-Value 

• 

Contingency  Coef. 

.586 

Cramer's  V 

.512 

It  appears  that  the  first  category  (W)  has  many  people  who  received  an  A  in  the 
class.  This  first  category  can  be  described  as  a  pattern  where  the  user  returned  to  most 
screens  at  least  3  times.  Also,  of  interest  is  that  the  majority  of  the  people  who  got  a  B  in 
the  course  fall  into  category  X.  This  category  can  be  described  as  a  pattern  where  the 
majority  of  the  screens  in  the  program  were  visited  only  once.  This  result  points  toward  an 
interesting  question  for  future  research.  That  is,  does  the  pattern  where  the  learner  returns 
to  the  same  screen  many  times  result  in  better  learning?  And,  if  this  statement  is  true,  can 
a  system  be  designed  that  adjusts  the  level  of  user  control  based  on  this  interaction  pattern. 
There  were  not  enough  people  who  received  a  C  or  fell  into  the  last  two  categories  to  conclude 
anything  about  them. 
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Figure  5.  An  example  of  category  W  created  by  N2. 


Figure  7.  Summary  table  and  observed  frequencies  for  the  only  categorization  scheme  that 
was  significant  for  grade  as  based  on  treatment  group. 

Summary  Table  for  Group,  N2  Observed  Frequencies  for  Group,  N2 


Num.  Missing 

0 

W 

X 

Y 

z 

Totals 

DF 

6 

Browse 

3 

6 

0 

1 

10 

Chi  Square 

9.992 

Index 

3 

8 

1 

1 

13 

Chi  Square  P-Value 

.1250 

Study 

11 

3 

0 

1 

15 

G-  Squared 

• 

Totals 

17 

17 

1 

3 

38 

G-Squared  P-Value 

• 

Contingency  Coef . 

.456 

Cramer's  V 

.363 

«/  u  o 


Figure  6\  An  example  of  category  X  created  by  N2. 


A  related  question  to  the  one  described  above  is  how  do  the  categories  generated  by 
subject  N2  relate  to  the  experimental  treatments  in  the  original  Nelson  (1991)  study? 
Figure  7  depicts  the  results  of  this  analysis.  While  the  result  is  not  significant,  a  pattern 
similar  to  grade  can  be  seen.  That  is,  there  are  a  large  number  of  subjects  that  are  both  in 
the  study  treatment  and  fall  into  the  pattern  where  the  user  returned  to  most  screens  at 
least  3  times  (category  W).  Likewise,  the  pattern  where  the  majority  of  the  screens  in  the 
program  were  visited  only  once  (category  X)  is  well  represented  by  the  Browse  and  Index 
treatments.  While  it  is  premature  to  make  any  kind  of  definitive  statement  about  whether 
these  patterns  are  at  all  related  to  learning,  it  does  hold  some  promise  for  a  more  rigorous 
experimental  examination. 

Chi-squared  analyses  were  performed  on  the  categoric  generated  by  the  subjects  in 
this  study  with  the  treatment  groups  in  the  Nelson  study.  The  results  indicated  that  all  of 
the  Instructional  Technology  doctoral  students'  categories  were  significantly  related  to  the 
treatment  groups.  This  is  not  surprising  given  that  the  doctoral  students  were  aware  of  the 
groups  and  from  which  group  each  image  came.  There  were  also  two  subjects  that  were  not 
informed  of  what  the  images  represented  that  resulted  in  significant  categories.  These 
categories  were  generated  by  the  teacher  group  in  the  current  study  and  the  results  an1 
depicted  in  Figures  8  and  9. 


Figure  S,  Summary  table  and  observed  frequencies  for  significant  categorization  scheme 
based  on  treatment  group. 


Summary  Table  for  Group,  N4 


Observed  Frequencies  for  Group,  N4 


Num.  Missing 

0 

S 

T 

U 

V 

Totals 

DF 

6 

Browse 

2 

4 

0 

4 

10 

Chi  Square 

13.218 

Index 

2 

6 

2 

3 

13 

Chi  Square  P-Value 

,0397 

Study 

1 

0 

4 

10 

15 

G-Squared 

• 

Totals 

5 

10 

6 

17 

38 

G-Squared  P-Value 

• 

Contingency  Coef . 

.508 

Cramer's  V 

.417 

Figure  9.  Summary  table  and  observed  frequencies  for  significant  categorization  scheme 
based  on  treatment  group. 


Summary  Table  for  Group,  N6 


Observed  Frequencies  for  Group,  N6 


Num.  Missing 

1 

O 

Q 

R 

Totals 

DF 

4 

Browse 

3 

2 

4 

9 

Chi  Square 

10.972 

Index 

4 

4 

5 

13 

Chi  Square  P-Value 

.0269 

Study 

0 

1 

14 

15 

G-Squared 

• 

Totals 

7 

7 

23 

37 

G-Squared  P-Vaiue 

• 

Contingency  Coef. 

.478 

Cramer's  V 

.385 

For  subject  N4f  categories  T  and  V  seem  to  define  people  in  the  treatment  groups 
best.  That  is,  category  T  does  not  include4  anyone*  from  the  study  treatment  group. 
Similarly,  category  V  accounts  for  the  majority  of  the  study  group  (10  out  of  1 7).  The 
categorization  scheme  used  by  subject  N4  is  based  on  tin*  shading  of  nodes  (junt  as  subject 
N2).  In  order  to  be  included  in  category  T,  the  picture  should  contain  "no  morn  than  2 
different  shades."  Category  V  for  subject  N4  was  that  there  were  "no  more!  than  4  different 
shades  of  boxes."  Upon  examining  the  nature  of  the1  images  for  the  category  T,  we  noticed 
that  most  all  of  these  images  had  the  2  shades  of  light,  gray  and  white  (all  but  2  of  them). 
This  is  not  an  interaction  pattern  that  would  indicate  that  these  learners  were  returning  to 
screens  to  re-examine  information.  On  the  other  hand,  the  V  category  mIiows  a  pattern  of 
behavior  that  certain  screens  are  visited  once,  others  twice,  etr.  While  it  is  pure  conjecture 
on  our  part,  this  would  indicate  that  the  users  in  this  category  were  judging  the  value  of  the 
information  and  returning  to  screens  that  were  important  and  not  returning  to  screens  that 
were  not. 

Subject  N6  used  a  categorization  scheme  that  was  based  on  both  frequency  of 
returning  to  a  screen  (shade)  and  the  number  of  screens  visited  (the  number  of  nodes  on  the 
image).  Categories  O  and  Q  can  be  collapsed  into  a  single  category  where  there  were  "no 
dark  gray  or  black  boxes  and  6  or  fewer  boxes."  Category  li  would  be  the  opposite  (the 
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image  contained  dark  gray  or  black  boxes  and  it  had  more  than  6  boxes).  It  is  interesting  to 
note  that  frequency  of  visiting  screens  is  such  an  important  attribute  (it  is  an  aspect  of  each 
of  the  categories  described).  Future  research  ought  to  examine  this  navigation  pattern  more 
closely. 

Conclusions 

The  three  methods  for  analyzing  user  interactions  with  hypermedia  systems 
described  above  have  been  successfully  employed  to  compare  the  patterns  of  navigation 
exhibited  by  groups  of  users  in  several  studies.  The  last  analysis  seems  to  also  provide 
some  potential  for  further  research  more  directly  related  to  navigation  patterns  as  they 
relate  to  learning.  The  analysis  of  peoples  interactions  with  hypermedia  systems  is  quite 
complex.  The  capability  to  visualize  this  interaction  allows  a  researcher  the  ability  to 
examine  many  different  attributes  simultaneously  within  the  image.  The  results  indicate 
that  there  appears  to  be  merit  to  this  type  of  analysis. 

At  this  time,  the  methods  in  this  paper  appear  promising  for  further  research  into 
usability  studies  of  hypermedia  systems,  as  well  as  more  basic  research  into  various 
theories  of  human -computer  interaction.  The  latter  research  will  also  require  additional 
methodologies  in  order  to  obtain  the  rich  descriptions  of  interaction  with  hypermedia  that  is 
necessary  to  further  develop  theories  and  principles  for  the  design  of  such  systems. 
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Technology  has  changed  the  way  we  view  education  and  the  delivery  of  instruction. 
Distance  education  is  no  exception  when  it  comes  to  technology  advancement  and 
change.  Distance  education  (DE)  is  described  as  an  alternative  delivery  of 
instruction  (Relaford,  1988);  instruction  that  is  delivered  by  a  method  other  than 
through  the  traditional  classroom  setting.  It  can  be  defined  as  "formal  instruction 
in  which  a  majority  of  the  teaching  function  occurs  while  educator  and  learner  are  at 
a  distance  from  one  another"  (Verduin  &  Clark,  1991,  p.  19).  "Distance  learning 
refers  to  the  use  of  telecommunications  (audio  and  video)  to  bring  people  together  for 
the  purpose  of  learning"  (Brownell,  1992,  p.  510). 

Technology  advancement  has  allowed  DE  delivery  systems  to  provide  students 
needed  instructional  opportunities  that  would  otherwise  pass  them  by  (Barker, 
1992).  There  are  several  types  of  DE  delivery  systems  that  promote  distance 
learning  (see  appendix  A).  They  include  telephone  audioconferencing,  computer- 
based  messaging,  audiographic  conferencing,  interactive  television,  satellite 
broadcast,  and  others  (Clark,  1989).  Each  of  these  systems  involves  specific 
technology  that  allows  for  the  transfer  of  educational  information.  This  information 
provides  the  learner  with  instructional  elements  that  are  necessary  to  help  learning 
occur. 

Research  indicates  that  these  systems  are  effective  in  producing  learning.  Each  has 
advantages  and  disadvantages  when  used  to  deliver  instruction.  Our  focus  here  is 
to  examine  the  use  of  interactive  television  as  the  DE  instructional  delivery  medium. 
A  review  of  research  and  related  literature  on  distance  education  delivered  primarily 
through  interactive  television  will  help  determine  what  practices  in  course  design 
produce  effective  DE,  what  conditions  of  learning  are  associated  with  effective  DE 
courses  and  what  learner  characteristics  are  most  important  in  designing  effective 
DE  programs. 

Interactive  Television  Effectiveness 

Interactive  television  technology  as  a  DE  delivery  system  involves  two-way 
communication  between  an  originating  classroom  and  remote  classroom  sites. 
Wilson  (1991)  states  that  such  systems  "provide  immediate  feedback  and 
participation  in  interactive  instructional  processes  through  the  use  of  alternative 
electronic  communications."(p.  27)  Video  and  audio  signals  are  transmitted  to  and 
from  the  originating  and  remote  sites  via  paths  such  as  telephone  lines,  microwave 
systems,  and  satellite.  This  telecommunication  technology  provides  enhanced  DE 
instructional  delivery  because  learners  from  every  site  can  see  and  interact  with  the 
instructor  during  classroom  meetings  and  instructional  presentations.  According  to 
Moore  and  McLaughlin  (1992)  courses  offered  by  such  a  system  at  the  St.  Cloud 
State  University  indicate  that  "students  achieve  at  about  the  same  rate  and  level 
as  do  students  in  on-campus  sites.  As  a  matter  of  fact,  students  at  remote 
locations  tend  to  do  a  bit  better  on  their  class  examinations.M(p.  75) 
Robinson  (cited  in  Moore  &  Thompson  et  al,  1990)  presented  results  from  a  five  year 
study  of  the  two-way  Carroll  Instructional  Television  Consortium.  Data  indicated 
that  "student  achievement  levels  are  comparable  to  those  found  in  live 
classrooms. "(p.  68)  Morehouse,  Hoaglund  &  Schmidt  et  al  (cited  in  Relaford,  1988) 
presented  results  on  the  effectiveness  of  interactive  television  from  an  analysis  of 
the  Minnesota  system  that  indicated: 
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"Instructional  television  students  enjoyed  learning  and  achieved  at  a  level  similar  to 
students  in  traditional  classes.  There  was  no  evidence  of  statistically  significant 
differences  in  achievement  and/or  in  the  rate  of  learning. "(p.  27) 

This  and  other  research  findings  indicate  that  two-way  interactive  television  is  an 
effective  delivery  system  foi  promoting  learning.  Other  factors  also  influence  the 
learning  experience  resulting  from  the  use  of  two-way  interactive  television. 

Interactive  Television  Advantages 

According  to  Alaska  University  (1989a)  instructional  television  offers  many  potential 
advantages,  including  visual  imagery,  motion,  motivation/Interest,  persuasion, 
illustration  of  major  points,  summarization  of  key  concepts,  presenting  otherwise 
unavailable  experts  or  resources  and  others.  It  i«  particularly  effective  in  delivering 
primary  content  and  is  most  effective  when  combined  with  interactive  activities. (p. 
4) 

Hudspeth  &  Brey  (cited  in  Relaford,  1988)  suggest  that  interactive  two-way  audio 
and  video  classes  provide  the  best  alternative  to  the  traditional  classroom  and  that 
they  have  these  advantages: 

*  It  provides  a  means  to  bring  master  teachers,  upper  level  courses  and 
graduate  courses  to  smaller  colleges. 

*  It  is  an  excellent  tool  for  providing  choices. 

*  It  is  a  means  for  providing  enrichment  and  advanced  offerings  to  students. 
Barker  (1990)  includes  these  advantages  for  two-way  interactive  instruction: 

*  Two-way  full-motion  video  is  possible  between  all  sites;  students  can  see 
the  teacher  as  well  as  other  students  at  different  sites,  and  the  teacher  can 
see  all  students  at  all  sites. 

*  Most  systems  presently  in  operation  are  small  networks  that  promote  local 
control  of  the  teacher  and  curriculum  and  maintain  an  overall  class  size. 

*  Open-line  microphones  allow  for  full  teacher-  student  and  student-to- 
student  audio  interaction.  That  is  students  can  interact  audibly  not  only 
with  the  teacher  but  also  with  students  at  other  sites. 

These  advantages  help  interactive  television,  as  an  DE  delivery  medium,  to  be 
effective  in  aiding  the  learning  process.  However,  learning  occurs  because 
information  has  been  transmitted  to  and  from  the  learner  so  that  new  knowledge  is 
acquired.   Because  the  information  transferred  to  and  from  the  student  is 
important  to  the  learning  process,  DE  courses  require  the  implementation  of 
instructional  design  principles  (Moore  ot  al,  1990;  Texas  Education  Agency,  1987)  to 
insure  proper  course  design  and  planning  and  ii  successful  teaching/learning 
experience.  DE  instructional  planning  is  complex  and  involves  expertise  in 
designing,  planning  and  delivering  of  course  materials,  and  evaluating  the  DE 
course  ho  that  appropriate  changes  can  be  made  to  improve  future  course  delivery. 


Instructional  Design  in  Distance  Education 

The  effectiveness  of  DE  instructional  delivery  will  depend  upon  the  planning  of  the 
instructional  materials  and  activities  to  be  used  in  the  delivery.  The  use  of  a 
systematic  instructional  design  (ID)  process  will  assure  that  the  delivered 
instruction  will  be  effective  and  successful  in  promoting  learning.  This  systematic 
process  of  planning  instruction  will  aid  the  educator  or  instructional  designer  in 
planning  and  developing  such  activities  and  instructional  materials. 

Dick  and  Carey  (cited  in  Gagne,  Briggs  &  Wager,  1988)  describe  an  ID  model  that 
includes  nine  stages: 

(1)  identify  instructional  goals; 

(2)  conduct  an  instructional  analysis; 

(3)  identify  entry  behaviors  and  learner  characteristics; 

(4)  write  performance  objectives; 

(5)  develop  criterion-referenced  test  items; 

(6)  develop  an  instructional  strategy; 

(7)  develop  and  select  instructional  materials; 

(8)  design  and  conduct  a  formative  evaluation  with  possible  need  for 
revision;  and 

(9)  design  and  conduct  a  summative  evaluation. 

The  Alaska  University  (1989b)  suggests  a  four  step  ID  model  that  includes  design, 
development,  evaluation  and  revision.  The  design  stage  includes: 

*  Determining  the  need  for  instruction. 

*  Analyzing  the  intended  audience. 

*  Establishing  instructional  goals. 

The  development  stage  includes: 

*  Deriving  specific  objectives  for  goals. 

*  Creating  a  content  outline. 

*  Reviewing  existing  materials. 

*  Organizing  content. 

*  The  selecting  and  developing  of  materials  and  methods. 
The  evaluation  stage  includes: 

*  Reviewing  goals  and  objectives. 

*  Developing  an  evaluation  strategy. 

*  Collecting  and  evaluating  data, 

*  Determining  recommended  revisions  based  on  the  data 
analysis. 

And  finally,  the  revision  stage  allows  for  the  development  and  implementation  of  a 
revised  instructional  plan. 
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lieliiirli,  MdUfinln,  iiliti  Russell  (cited  in  Moller,  1991)  provide  the  ASHUHK  model  for 
philinniH  mngraiilH  1 1  nil  utilize  DE  technologies.  It  is  primarily  uboiJ  by  tint 
individual  iiiHt  rlii?(fir  for  planning  daily  classroom  use  of  media.  The  AHSUUK  model 
It 4iM  mix  Htti|m: 

I  Analyze  l*earnerH 
U   Hi  ill  <i  Objectives 

II  *  Heleet  Media  nml  Materials 

I  I  ItlllKn  MiitiirtnlH 

li   Ueipiire  l«nartmr  Performance) 

II  Evaluate  anil  Revise 

llayni'H  ami  Dillon  (MHIi!)  apply  an  instructional  system  design  (ISO)  model 
adapted  from  t  limne  anil  ItfigM"  that  can  help  design  instruction  consisting  of 
several  interrelated  activities.  These  include: 

(1)  thi)  analyMiH  uf  learner  charaetcristics, 

(2)  thi'  determination  of  instructional  objectives, 
the  selection  of  strategies, 

(4)  the  design  of  media, 

(f>)  the  implementation  of  infraction,  and 

(fi)  the  evaluation  nf  the  outcomes  linked  to  the  specific 

objectives  and  strategies. 

ID  models  represent  the  framework  that  is  used  to  guide  the  entire  process  of 
designing  instructional  systems  (Heinich,  Molenda,  and  Russell,  1989).  Each  of  the 
ID  models  included  here  can  aid  in  providing  the  framework  necessary  to  plan 
effective  instruction  and  activities.  There  are  many  other  ID  models.  Appendix  C 
contains  a  detailed  description  of  an  instructional  design  model  adapted  from  the 
ASSURE  model 

Environmental  Characteristics  of  Distance  Education 

An  understanding  of  the  characteristics  belonging  to  the  distance  education 
environment  will  aid  the  design  and  planning  processes.  Keegan  (1990)  cites  five 
characteristics  of  distance  education: 

*  The  quasi-permanent  separation  of  teacher  and  learner  throughout  the 
length  of  the  learning  process  (this  distinguishes  it  from  conventional  face-to- 
face  education). 

*  The  influence  of  an  educational  organization  both  in  the  planning  and 
preparation  of  learning  materials  and  in  the  provision  of  student  support 
services  (this  distinguishes  it  from  private  study  and  teach-yourself 
programs). 

*  The  use  of  technical  media  -  print,  audio,  video,  or  computer  -  to  unite 
teacher  and  learner  and  carry  out  the  content  of  the  course. 
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*  The  provision  of  two-way  communication  so  that  the  student  may  benefit 
from  or  even  initiate  dialog  (this  distinguishes  it  from  other  uses  of 
technology  in  education). 

*  The  quasi-permanent  absence  of  the  learning  group  throughout  the  length 
of  the  learning  process  so  that  people  are  usually  taught  as  individuals  and 
not  in  groups,  with  the  possibility  of  occasional  meetings  for  both  didactic 
and  socialization  purposes,  (p.  44) 

According  to  the  Office  of  Technology  Assessment  (1989), 

"Students,  teachers,  course  material  and  presentation,  and  interaction  are  all 
affected;  distance  learning  creates  a  new  context  within  which  the  education  process 
and  student-teacher  interaction  take  place.  Old  styles  of  teaching  and  learning  may 
not  be  most  appropriate  or  effective  when  mediated  by  telecommunications 
technologies.. ..In  designing  distance  courses  or  modules,  teachers  and  instructional 
designers  have  had  to  find  ways  to  restructure  interactivity."(p.  93) 

The  Texas  Education  Agency  (1987)  states  that  "because  of  the  intercession  of 
distance  and  devices  between  the  teacher  and  the  student,  new  instructional 
techniques,  varied  uses  of  material  and  modification  of  class  management  may  be 
appropriate. "(p.  5) 

Tin'  consideration  of  them1  characteristics  during  the  design  and  planning  stages 
will  help  the  educator  or  instructional  designer  plan  for  the  unique  opportunities 
made  available  with  the  Dli  communication  technology  being  used  to  deliver 
instruction. 


Conine  Design  and  Planning 

Design  and  assessment  are  key  areas  where  distance  educators  and  educational 
technologists  can  influencing  how  students  go  about  their  studies  (Morgan,  1991,  p. 
It)).  Important  considerations  should  include  the  delivery  system,  available 
resources,  course  content,  the  target  learner  audience,  their  learning  styles,  and 
their  ability  to  engage  in  the  learning  process.  Instructional  experiences  can  be 
carefully  designed  and  developed  so  that  they  can  be  used  by  a  wide  variety  of 
learners  possessing  similar  learner-related  characteristics  (Dwyer,  1990),  The 
course  planning  process  can  involve  many  people  who  are  specialists  in  certain 
fields.   Members  of  such  a  planning  team  might  include  subject  authors, 
instructional  technologists,  illustrators,  media  specialists,  librarians,  photo- 
librarians,  and  editors  (Moore  and  Thompson  et  al,  1990,  p.  4).  Brinkley, 
Pavlechko  &  Thompson  (1991)  suggest  a  design  team  consisting  of  the  course 
instructor,  an  instructional  designer,  a  producer/director,  a  graphic  artist  and  an  on- 
air  director.  However,  the  individual  instructor  may  be  the  entire  planning  team 
and  provide  the  necessary  course  content  that  would  be  edited  and  published  for  a 
DE  course.  A  realistic  planning  team  for  a  typical  distance  education  course  might 
consist  of  the  instructor,  an  instructional  technologist  (instructiona)  designer),  and  a 
student  or  potential  student  of  the  course. 
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Development  of  course  content  can  begin  by  representing  it  in  a  manner  that 
enables  the  planning  team  to  develop  a  ''.ourse  description,  general  aims, 
instructional  goals,  units  and  lessons/  modules  to  drive  the  content."  (Brinkley, 
Pavlechko  &  Thompson,  1991,  p.  51)  Thus,  the  purpose  of  the  planning  team  is  to 
structure  academic  content  in  a  form  suitable  for  study  by  distant  learners  (Moore  & 
Thompson  et  al,  1990).  Instructional  design  principles  can  be  utilized  to  provide  for 
instructional  strategies  that  fit  the  medium  of  delivery. 

In  regard  to  planning  instruction,  Wolcott  (1993)  cites  three  studies  that  indicate 
that  college  teachers'  majors  concerns  include:  selecting  content  and  course 
materials,  covering  the  course  content,  class  processes,  on-going  planning 
activities,  students'  participation  in  class  discussions,  communicating  with 
students,  and  recognizing  student  characteristics, 

McGreal  (1991)  provides  a  set  of  guidelines  to  help  the  ordinary  teacher  produce 
instructional  materials  designed  for  distance  education  systems.  They  include: 

(1)  The  materials  should  be  simple  so  that  the  lonelier  and  students*  are 
clear  about  the  goals  of  each  lesson. 

(2)  The  materials  should  be  highly  structured  ho  that  the  objectives  of  the 
lesson  are  achieved. 

(3)  There  should  be  many  reviews  oft  he  material  taught  in  previous  lessons. 

(4)  Evaluations  should  take  place  frequently  to  ensure  that  sufficient 
learning  is  occurring. 

(5)  High  interest  content  material  should  he  found  which  can  lower  teacher 
anxiety,  stimulate  the  students  participating  in  the  rmirxc,  and  develop  a 
rapport  with  them. 

(6)  The  materials  must  he  relevant  to  the  particular  needs  of  the  students. 

(7)  Extension  activities  should  he  marie  available  fur  students  who  need 
more  content  to  aid  the  learning  process. 

(8)  This  material,  although  highly  structured,  must  be  clearly  perceived  by 
the  teacher  to  be  non-restrictive  to  his/her  touching  style. 

(9)  The  material  should  he  activity  oriented  in  order  to  divert  teachers  away 
from  lecturing  to  students  and  promote  active  student  learning  during  class 
sessions.(pp.  H-9) 

Parker  and  Monson  (cited  in  Moore  &  Thompson  et  al,  1990)  contend  that  specific 
techniques  should  be  incorporated  into  the  course  development  process.  The 
techniques  include  humanizing,  participation,  message  style  and  feedback. 

"Humanizing  refers  to  the  creation  of  an  accepting  environment  which  breaks  down 
the  barrier  of  distance  and  generates  feelings  of  rapport  between  teacher  and 
students.  Participation  deals  with  the  extent  of  interaction  among  participants  in 
the  interactive  situation.  Message  style  refers  to  ways  of  enhancing  the  interest  and 
appeal  of  a  presentation.  Planning  for  short  instructional  segments,  varying  tone  of 
voice  and  volume,  and  supplementing  programs  with  visual  aids  maintain  the 
interest  and  attention  of  the  students.  Feedback  allows  instructors  to  determine  if 
their  presentations  were  clear  and  effective.  Both  verbal  and  written  feedback 
should  be  obtained,  and  can  include  questionnaires,  interviews,  or  group  reports." 
(p.  21) 
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Allen  and  Carl  (1988)  list  seven  characteristics  necessary  for  the  creation  of  a 
successful  distance  learning  program  (see  Appendix  D  for  more  information).  This 
model  was  adapted  from  guidelines  found  in  "Guide  to  Distance  Learning  as  an 
Alternative  Delivery  Procedure"  published  by  the  Texas  Education  Agency.  The 
characteristics  include: 

(1)  high  standards  are  established  for  their  teachers  and  mediators  (student  site 
contact  persons;  also  called  proctors,  monitors,  aides,  etc.) 

(2)  consistently  train  and  retrain  staff 

(3)  carefully  evaluate  teaching  sessions  and/or  the  course  packages  before 
distribution  by  telecommunication  or  other  means 

(4)  ensure  that  there  are  provisions  for  easy  communications  between  students, 
instructors  and  proctors 

(5)  develop  rigorous  and  valid  testing  programs  which  provide  rapid  response  times 
and  feedback  loops 

(6)  package  the  course  content 

(7)  provide  ready  accessibility  to  resources  such  as  mediators  and  ample 
audiovisuals,  print  and  other  media  for  student  use.  (Texas  Education  Agency, 
1987,  pp.  4-5) 

When  the  elements  included  here  are  considered  during  the  design  and  planning 
stages,  course  effectiveness  and  efficiency  will  surely  benefit.  The  design  and 
production  of  course  materials  requires  such  considerations  to  produce  effective 
delivery  through  DE  technology.  An  example  of  the  development  process  can  be  seen 
in  the  elements  included  in  Brinkley,  Pavlechko  &  Thompson's  (1991)  four  element 
team  approach.  They  include: 

*  Instruction 

*  Production 

*  Design 

*  On-Air  Direction 
Steps  of  the  approach  include: 

*  A  faculty  member  is  contracted  to  teach  using  distance 
education  technology, 

*  A  map  of  course  content  is  given  to  the  instructional  designer. 
The  designer  will  analyze  the  map  and,  if  necessary,  ask  the 
faculty  member  for  clarification. 

The  designer  then  gives  the  map  to  the  production  team  (the  map 
is  an  outline  representing  the  content  of  the  course). 

*  The  design/production  team  members  have  an  understanding 
of  the   course  content  and  are  prepared  to  discuss  it  with  the 
faculty  member. 

*  The  design/production  team  asks  the  instructor  questions  about 
the  content  of  the  course  (the  course  map)  and  the  faculty 
member  uses  a  computer  program  to  plan  the  first  lesson, 

*  The  teacher  designs  a  single  lesson  plan  consisting  of  five  parts 
and  gives  it  to  the  instructional  designer.  The  designer         will  discuss 
the  lesson  plan  with  the  faculty  member  and,  if         necessary,  ask 
him/her  for  clarification.  The  designer  then     givesthe  lesson  plan  to  the 
production  team. 


*  The  design/production  team  has  an  understanding  of  the  lesson 
plan  and  the  teaching  styles  and  strategies  the  instructor      will  use  for  the 
distance  education  medium. 

*  Specific  visuals  for  the  first  lesson  plan  are  discussed. 
Specifics  such  as  which  slides  and  videotapes  will  be  used  are 
decided  at  this  meeting  with  the  faculty  member,  design  and 
production  team. 

*  An  on-air  script  is  produce  by  the  producer  and  given  to  the 
on-air  director.  This  step  focuses  on  cues  that  the  director  needs 
when    operating  the  controls  that  deliver 

the  course  to  distant  sites  such  as  videotapes,  slides  and  graphics. 

*  The  faculty  member  reviews  the  media  for  the  single  lesson 
he/she  has  produced  and  makes  any  final  changes  to  the  media. 
The  remaining  lesson  plans  are  then  produced  tor  the 

course. (pp.  50-54) 

Cyrs  and  Smith  (1991)  provide  design  and  production  steps  needed  to  produce 
interactive  study  guides  for  interactive  television  courses.  Such  guides  are  described 
as  highly  organized  sets  of  "student  notes,  graphics  pictures,  graphs,  charts  and 
activities  which  are  used  in  conjunction  with  a  tclelecture."(p.  37)  The  primary 
purpose  of  such  guides  is  to  focus  the  learner  on  key  concepts  associated  with 
instructional  content.  Such  guides  allow  the  learners  to  concentrate  "on  the 
cognitive,  analytical  and  critical  thinking  skills  being  demonstrated  by  the 
instructor."(ibid.) 

Design  steps  for  developing  an  interactive  study  guide  include: 

*  Outline  the  content  of  one  telelecture. 

*  Select  a  short  instructional  segment  that  can  be  shown  as  a 
word    picture  (a  graphic        representation  of  an  idea,  concept, 
data,  geometric!  shapes,  clip  art,  symbols,  etc.). 

*  Identify  tho  main  concept  and  the  supporting  or  sub-concepts 
(or  examples  of  the  concept)  to  be  illustrated. 

*  Try  to  visualize  the  concept  and  sub-concepts  by  looking  for 
ways  to  turn  the  words  iato  pictures.  If  you  can  find  clip  art  or 
other    appropriate  graphics  use  them  and  show  relationships 
among  the  elements  by  using  arrows  and  lines. 

*  Concepts  that  cannot  be  represented  with  graphic  art  can  be 
placed  in  boxes  or  circles  and  arrows  and  lines  can  be  used  to 
indicate  the     element  relationships. 

*  Leave  out  some  of  the  key  concept  words  and  insert  a  blank 
line  so  that  students  will  have  an  opportunity  to  fill  in  notes  as 
you  lecture. 

*  Proceed  to  design  Word  Pictures  for  all  of  the  instructional 
segments  in  the  lesson,  being  careful  to  indicate  the  proper 
sequence  for  presentation,  (p,  38) 

Wolcott  (1993)  describes  the  planning  processes  utilized  by  faculty  members  who 
are  involved  in  distance  teaching.  Results  from  her  study  describe  term  planning, 
content,  and  the  extended  syllabus. 


Participants  (faculty)  approached  the  task  by  planning  at  the  course  or  term 
level.  Planning  took  the  form  of  a  time-consuming,  front-end  activity  rather  than  an 
on. going  one.. .essentially  packaging  the  course  into  an  extensive  syllabus,... The 
faculty  focused  on  selection  and  sequencing  of  content.  In  planning  the  course,  the 
faculty  were  concerned  with  two  dependent  tasks;  1)  defining  the  content,  i.e., 
determining  what  to  include,  and  2)  matching  content  with  the  time  available  in 
which  to  present  it.  These  two  tasks  were  embedded  in  the  central  planning 
activity,  the  development  of  the  course  syllabus.  The  faculty  made  their  planning 
decisions  tangible  in  the  form  of  a  course  syllabus.. ..The  syllabus,  in  most  cases, 
was  far  more  extensive  than  the  typical  one-  or  two-page  handout.  Commonly 
referred  to  as  the  "expanded,"  "enhanced,"  or  "extended"  syllabus,  these  syllabi  had 
as  their  base  the  traditional  class  syllabus  that  included  a  course  description, 
course  goals,  readings  and  assignments,  a  topic  outline,  and  grading  policies.  In 
addition  to  these  standard  items,  the  syllabi  also  contained  handouts  or  hard 
copies  of  overhead  transparencies,  study  questions,  reprinted  articles,  and  even 
extensive  original  essays.'Xpp.  28-29) 

Although  there  are  few  studies  that  investigate  the  planning  of  distance  teaching  in 
higher  education,  the  literature  to  date  provides  large  amounts  of  information  on  the 
design  and  development  of  distance  learning  courses.  Clearly,  the  designing  of  any 
DE  course  can  be  a  complex,  time  consuming  task.  There  are  many  pieces  to  fit 
togethn*  during  the  planning  process  that  aid  in  the  development  of  course 
production.  Other  considerations  of  course  development  include  the  specific 
dements  associated  with  the  distance  learner. 

Conditions  of  Learning 

The  preparation  (if  instructional  lessons  and  the  delivery  of  such  lessons  to  the 
distance  learner  involves  understanding  the  learner's  previously  acquired  knowledge 
and  learning  capabilities.  Lessons  should  be  prepared  to  meet  the  knowledge  levels 
of  tin1  learners  and  provide  the  best  method  to  transfer  important  information  to 
them,  (tagiio*  (cited  in  (Jey,  1990)  provides  nine  events  of  instruction  that  should  be 
considered  ia  the  instructional  delivery  process.  These  events  are  designed  so  that 
each  learner1*  natural  cognitive  processing  is  followed  and  new  knowledge  is 
acquired.  Lchhoii  material  should  he  designed  for  the  learners  who  are  to 
participate  in  the  instructional  setting.  Appendix  C  includes  the  nine  events  of 
instruction  in  detail.   In  targeting  the  adult  learner,  Knowles's  (cited  in  Wilson, 
1991)  andragngieal  model  makes  the  following  assumptions: 


*  Adults'  self-concept  moves  from  one  of  dependent  personality  to  self* 
directed  human  being. 

*  Adults  accumulate  a  growing  reservoir  of  experience  that  becomes  a  rich 
source  of  learning. 

*  Readiness  to  learn  becomes  increasingly  oriented  to  their  developmental 
tasks  of  their  social  roles, 

*  Shift  in  immediate  application  as  their  orientation  of  learning  shifts  from 
subject  centeredness  to  performance  centeredncss.(p.  42) 


Wilson  (1991)  concludes  that  students  learn  when  all  of  the  above  conditions  are 
considered  plus  the  following  conditions  which  are  more  specific  to  Distance 
Education: 

*  Self-pacing  is  allowed; 

*  Feedback  is  in  the  "fastest  turn -around  time  that  is  feasible"; 

*  Interaction  with  the  institution  is  friendly  and  nonthreatening; 

*  Curriculum  is  structured  to  guide  the  student  in  linking  instruction  with 
previous  knowledge  and  interests;  and 

*  They  are  self-motivated.(p.  42) 

The  transfer  of  instruction  via  interactive  television  should  be  done  in  a  manner 
that  is  effective  and  efficient  in  providing  for  the  learners'  needs.  It  should  utilize  all 
available  resources  to  achieve  this  goal.  The  use  of  teaching  methods  can  aid  in  the 
learning  process.  The  message  that  is  intended  for  the  learner  is  transferred 
through  this  medium.  Heinich,  Molenda  &  Russell  (1989)  include  eight  methods  to 
deliver  the  instructional  message.  They  are  presentation,  demonstration,  drill-and 
practice,  tutorial,  gaming,  simulation,  discovery,  and  problem  solving.  Joyce  and 
Weil  (1986)  provide  a  variety  of  teaching  models  for  delivering  instruction. 
Determining  which  model  or  method  is  best  used  for  instruction,  depends  on  the 
learners  involved  and  the  content  of  the  lesson.  Information  concerning  learner 
characteristics  and  learning  styles  is  important  in  planning  and  delivering 
instruction  when  using  such  methods. 

Learner  Characteristics  and  Learning  Styles 

The  distance  learner  perceives  and  assimilates  information  delivered  through  the 
DE  system.  How  that  information  is  perceived,  depends  upon  lesson  content  and 
structure,  content  delivery,  and  the  learners'  learning  styles.  Learning  styles  play 
an  important  role  during  this  process.  Gee  (1990)  identifies  several  learning  stylus. 
They  include  generalizing,  exploratory,  systematic,  relationships,  verbal,  and  visual. 

Learning  style  inventory  measurements  are  helpful  in  determining  learning  styles. 
The  Canfield  learning  styles  inventory  (LSI)  is  one  measurement  instrument  that 
determines  learning  styles.  The  Canfield  LSI  measures  the  conditions  of  learning, 
the  content  of  learning,  the  mode  of  learning,  and  student  expectations  in  a  learning 
situation  (Roylan  &  Kerstiens,  Eds.,  1989).  According  to  Boylan  &  Kerstiens 
(ibid,),  the  Canfield  LSI  includes  several  specific  elements  associated  with  the  four 
dimensions  mentioned.   Under  the  category  of  conditions  of  learning  these  student 
preferences  are  measured: 

1.  Affiliation  •  pleasant,  friendly,  and  warm  relations  with  other  students  or 
faculty; 

2.  Structure  •  orderly,  logical,  and  well-defined  goals,  objectives,  and  study  plans; 

,\.  Achievement  -  independence,  self-determined  goals  and  objectives  in  relation  to 
perceived  skills  and  interests;  and 
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4.  Kmincnee  -  competition,  knowledge  (if  one's  own  performance  in  relation  to 
other's,  need  fur  control  or  authority. 

Under  the  category  of  Content,  flu*  itml ruim*iit  measures  student  preferences  for 
working  with  various  hoi'Im  of  content.  Thiw  root  on  t  sub -categories  include: 
numeric,  (pialitativc,  (working  with  words  or  language),  inanimate  (working  with 
things),  and  working  with  people. 

Canfield  agrees  with  < ingne's  notion  of  "Channel  efficiency,"  the  idea  that  in  every 
individual,  Home  channels  of  perceiving  and  processing  information  are  more  efficient 
than  others  (MMt7).  As  a  result,  his  instrument  also  measures  students'  preferred 
mode  of  learning.  The  catenaries  under  this  heading  are:  listening,  reading,  iconics 
(learning  through  illustrations,  movies,  slides,  graphs,  and  pictures,  etc.),  and  direct 
experience, 

Finally,  the  Canfield  LSI  assesses  student  expectations  of  learning  -  i.e.,  their 
anticipated  level  of  performance.  The  levels  of  anticipation  include  outstanding  or 
superior  performance,  good  or  above-average  performance,  average  or  satisfactory 
performance,  and  below-average  or  unsatisfactory  performance. 

The  instrument  measures  these  categories  through  thirty  items  in  which  the 
students  are  asked  to  rank  order  their  preferences  among  four  choices. (pp.  15-16)* 

Reliability  of  the  Canfield  LSI  instrument  was  established  to  be  high  and,  in 
addition,  validity  showed  "several  statistically  significant  differences"  among  all 
pairs  of  groups  tested  (ibid.).  Tamaoka  (1985)  generalizes  that  Mby  using  Canfield's 
LSI,  it  may  be  possible  to  compare  students1  learning  styles  with  teachers1 
instructional  style."  An  additional  instrument  designed  by  Canfield  provides  an 
assessment  of  instructional  style  to  correspond  to  his  LSI.   Brookfield  (who  cites 
Witken  and  Pratt  in  Wilson,  1991)  has  examined  the  relationship  between  cognitive 
styles  and  self- directed  learning.  Researchers  have  classified  individual  learners  as 
Field  Independent  (FI)  or  Field  Dependent  (FD).  FI  learners  are  those  who  prefer 
"solitary  situations  and  self-defined  goals,  strategies  and  reinforcement."(p.  43)  FD 
learners  are  those  who  prefer  "group  situations,  externally  defined  goals  and 
reinforcement,  and  explicit  instructions  or  definitions."  (ibid.)  The  literature 
indicates  that  the  FI  learner  is  more  likely  to  succeed  at  distance  education  (Wilson, 
1991).  Applying  interactive  television  in  a  manner  that  provides  learning 
opportunities  for  both  FI  and  FD  learners,  will  provide  learning  opportunities  for 
both  learning  groups. 

Identifying  learner  characteristics  and  the  determination  of  learners'  learning  stylos 
will  help  the  instructor  determine  what  instructional  stylc(s)  and  delivery  method(s) 
to  implement.  Course  content  can  be  delivered  so  that  the  students'  academic 
success  is  high.  Although  existing  research  in  distance  education  is  inconclusive 
concerning  the  effect  of  learning  style  preferences  on  achievement,  course  design  and 
assessment  will  play  a  critical  role  in  determining  what  learning  styles  are  best 
suited  for  the  information  and  instruction  delivered  via  interactive  television 
technology. 
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Teacher  Skills  and  Course  Delivery 


Distance  education  is  a  means  by  which  learners  can  receive  instruction  that  might 
not  be  available  to  them  by  traditional  course  delivery.  DE  systems  that  use  two- 
way  video  and  audio  are  utilized  to  provide  the  linking  instructional  medium  to 
many  remote  learners.  This  interactive  medium  is  an  effective  alternative  to  the 
traditional  classroom  setting.  Learners  are  able  to  interact  with  the  instructor  and 
other  learners  at  their  site  as  well  as  with  learners  at  participating  remote  sites. 
With  this  type  of  instructional  delivery  system,  the  teacher  must  be  able  to  provide 
effective  content  delivery  in  a  manner  that  uses  the  DE  system  such  that  the  learner 
receives  the  maximum  instructional  benefits  that  are  available. 

For  the  novice  distance  teacher,  familiarity  with  the  ITV  system  becomes  an 
essential  requirement.  Based  on  Thompson,  Simonson  &  Hargrave  (1992)  make 
these  recommendations  for  an  interactive  television  training  session: 

*  Provide  teachers  with  an  overview  of  the  technology  and  how  it  works. 

*  Provide  hands-on  guided  practice  on  the  use  of  ITV  technology. 

*  Incorporate  the  effective  elements  of  instruction  as  major  parts  of  the 
training  session. 

*  Have  periodic  follow -up  inservice  and  on-site  coaching  to  ensure  long  term 
training  benefits. 

*  Use  with  teachers  who  volunteer  for  ]TV  training. 

After  hemming  familiar  with  an  ITV  system,  teaching  skills  need  to  be  adapted  to 
fit  the  system.  The  DE  teacher  must  provide  the  necessary  interaction  with  the 
distance  learners  and  present  the  course  material  in  a  manner  that  involves 
learners  and  produces  learning.   Soaman  and  Fellenz  (cited  in  Wilson,  1991)  define 
adult  teaching  as:  "The  activity  through  which  the  teacher  or  learning  facilitator 
assists  the  adult  student  in  acquiring  new  knowledge  or  skills.,!(p.  44) 

The  teacher  is  responsible  for  providing  structure  and  feedback  to  the  distance 
learners.  Bra uc her  (cited  in  Moore  &  Thompson  et  al,  1990)  suggests  that 
"developing  a  friendly  atmosphere,  accurately  transmitting  feelings  by  tone  of  voice, 
carefully  selecting  words,  appropriately  using  silences, ...  and  thoughtfully 
integrating  new  students  are  skills  which  distinguish  effective  teachers  in  any 
setting  or  delivery  situation, "(p.  24) 

Wilson  (1991)  identifies  sever nl  skills  for  distance  teachers  from  the  literature:  1) 
imagines  what  the  students  need;  2)  inspires  the  students;  3)  encourages  them;  4) 
likes  people;  5)  is  alive;  6)  provides  feedback;  7)  motivates  students;  8)  tolerant;  9) 
cooperative;  10)  flexible;  11)  innovative;  12)  provides  two-way  written 
communication;  and  13)  establishes  personal  rapport. 

Specifically,  two-way  interactive  television  (ITV)  is  an  advanced  means  of  providing 
distance  education.  It  reflects  many  delivery  characteristics  of  the  traditional 
classroom  setting.  It  is  important  to  remember  that  there  is  a  missing 
personalization  with  remote  learners.  According  to  Tykwinski  and  l'oiilin  (cited  in 


Kolomeychuk  &  Peltz,  1991) 


"The  greatest  challenge  in  interactive  video  instruction,  or  meetings,  is  overcoming 
the  barriers  of  distance  and  technology  that  hinder  that  normal  personal 
interaction.  This  interaction  may  be  negatively  affected  by  the  novelty  or  fear  of 
technology,  number  of  participants  or  students  and/or  sites  included  in  an  event, 
decreased  ability  to  receive  non-verbal  cues  from  participants,  limited  opportunity 
for  after-meeting  or  after-class  discussions,  and  preconceived  notions  about 
television  viewing  (e.g.,  a  passive  activity). 

A  high  level  of  interaction  is  important  in  teaching  and  learning  and  in  meetings  in 
order  to  increase  the  attention  and  motivation  levels.  Interaction  can  also  be  used 
to  determine  whether  the  information  and  concepts  being  presented  are  properly 
received  and  interpreted.  It  is  recommended  that  leaders,  participants,  instructor 
and  students  become  confident  (through  exposure  and  training)  using  the 
technology,  limit  the  number  of  participants/sites,  evaluate  course  and  instructional 
design,  establish  interpersonal  rapport,  find  activities  that  promote  involvement, 
and  establish  new  questioning  strategies  to  promote  interaction  with  the 
audiences. "(p.  7) 

The  Office  of  Technology  Assessment  (1989,  p.  93)  states  that  "unless  distance 
learning  teachers  pay  close  attention  to  the  need  to  create  an  interactive 
environment  appropriate  to  the  technology,  students  can  and  will  tune  out." 
Johnstone  (cited  in  ibid.)  suggests: 

"The  best  way  to  learn  new  information  is  to  receive  it  while  in  an  active,  rather 
than  passive,  state  of  consciousness.... One  simple  method  the  instructor  can  use  to 
assist  the  learner.. .is  to  do  something  that  is  never  done  on  broadcast  television:  to 
talk  directly  to  the  distant  learner  and  require  a  response  at  the  very  beginning  of 
the  session."  (p.  93) 

The  Alaska  University  (1989c)  suggests  the  consideration  of  these  strategies  for 
teaching  at  a  distance: 

*  Adapt  instruction  to  meet  the  varied  needs  of  the  students,  the 
content,  and  the  limitations  of  the  delivery  system. 

*  Hold  a  pre-courso  audio  (or  ITV)  conference  to  increase  student 
familiarity  and  comfort  with  the  system. 

*  Familiarize  students  with  each  other  and  with  the  instructor: 
develop/distribute  student  unci  instructor  biographical  sketches,  make 
on-air  introductions,  have  students  state  their  names  and  locations  when 

they  address  the  group. 

*  Visit  the  different  sites  during  the  course  ,  meeting  individually  with 
students. 

*  Maintain  phone-in  office  hours  so  students  can  call  collect 

*  Develop  skills  to  facilitate  students'  learning  on  their  own  and  in  concert 
with  other  students  at  a  distance. 


*  Work  together  to  minimize  and  rectify  technological 
prohlems.(p.  3) 

Other  strategies  include:  a  welcome  letter,  a  clearly  written  and  complete  syllabus, 
course  materials  in  student  hands  on  time,  quick  return  of  student  work,  and  a 
willing  and  obvious  desire  to  meet  student  needs.(Alaska,  1990b,  p.  4) 
Additionally,  the  Learn  Alaska  Network  (cited  in  Office  of  Technology  Assessment, 
1989)  identifies  distance  teachers'  needs  for  assistance  is  such  areas  as: 

*  the  amount  of  time  needed  to  prepare  and  teach  distance  delivered 
courses, 

i 

*  methods  to  establish  and  maintain  effective  communication  with  distant 
students, 

*  experiences  with  other  faculty  members, 

*  strategies  for  adding  visual  components  to  audio  courses, 

*  planning  and  management  of  organizational  details  involved  in  distance 
delivery,  and 

*  strategies  to  encourage  group  cohesion  and  student  motivation. 

Communication  between  teacher  and  student  is  important.  There  are  several 
channels  that  can  be  used  to  communicate  with  the  learners  located  at  a  distance. 
They  include  the  telephone,  the  mail  or  postal  system,  computer  messaging  systems 
(bulletin  hoards  and  the  like),  and  in  the  case  of  an  ITV  delivery  system,  the  class 
meeting  itself.  The  use  of  computers  to  supplement  the  communication  between 
teacher  and  learner  has  distinct  advantages.  The  use  of  such  systems  allows  the 
students  to  move  at  their  own  pace  and  receive  feedback  from  the  teacher  in  a 
timely  fashion. 

Molmberg  (1990)  states: 

"The  possibility  to  use  micro-computers,  modems  and  telephone  communication,  so- 
called  electronic  mail,  to  attain  immediate  reception  both  of  students'  assignment 
papers  etc.  and  of  tutors'  correction  and  comments  will  undoubtedly  bring  about 
considerable  improvement.. ."(p.  16). 

Computer- mediated  communication  (CMC)  has  the  potential  to  favorably  affect  the 
course  quality,  attitudes,  and  motivational  levels  of  students.  The  convenience  of 
CMC  enhances  distance  education  effectiveness  and  provides  flexibility  and  other 
advantages  in  the  teacher-student  and  student-student  communication  processes 
(Boston,  1992). 

Communication  between  teacher  and  student  must  occur  in  a  manner  that 
encourages  personalization,  motivation,  and  understanding.  Effective  DE 
instructional  delivery  needs  as  much  personalization  as  is  necessary  to  provide  the 
distance  learner  with  every  possible  opportunity  to  learn  and  feel  comfortable  with 


the  DE  technology.  Existing  research  provides  a  list  of  guidelines  for  personalizing 
the  ITV  classroom.  ITV  teachers  should: 

*  Emphasize  and  encourage  active  student  participation. 

*  Meet  face-to-face  (if  at  all  possible)  with  remote  site  groups  prior  to  the 
first  scheduled  class. 

*  Schedule  formal  and  informal  face-to-face  meetings  with  the  class  during 
the  course  term. 

*  Teach  to  the  camera.  This  gives  the  remote  student  the  impression  of  eye 
contact  which  is  critical  to  personalization. 

*  Travel  to,  and  teach  from,  each  remote  site  if  possible. 

*  Devise  formative  evaluation  techniques  in  order  to  assess  the  success  of 
the  class  as  it  is  being  taught.  (Thompson,  Simonson-  and  Hargrave,  1992, 
p.  42) 

The  literature  and  existing  research  on  teacher  skills  and  course  delivery  includes 
an  extensive  amount  of  information  on  instructional  delivery.  Personalizing  course 
delivery  is  seen  as  an  essential  need  in  the  distance  learning  environment. 
Additional  communication  channels  are  important  to  providing  for  the  distance 
learners'  needs.  Familiarity  with  the  ITV  system  is  important  for  both  the  distance 
teacher  and  learner.  To  ensure  that  course  delivery  is  as  effective  as  possible, 
teachers  need  to  evaluate  and  assess  course  delivery. 

Course  Assessment 

Course  assessment  must  not  be  overlooked  during  the  course  design  and  planning 
stage.  It  is  addressed  here  and  uses  the  framework  provided  in  previous  sections. 
A  primary  purpose  of  course  assessment  is  *,o  determine  if  goals  and  objectives  that 
have  been  set  for  the  course  have  been  achieved.  This  will  help  determine  if  the 
instructional  methods  chosen,  and  selected  instructional  media  and  materials  are 
providing  the  means  necessary  to  achieve  such  goals  and  objectives. 

Assessment  strategy  will  include  formative  and  summative  evaluation  development. 
Formative  evaluation  will  provide  an  on-going  evaluation  process  that  originates 
from  the  development  of  course  materials  and  content  delivery.  Determination  of 
problem  areas  within  course  materials  and  content  delivery  will  indicate  revision 
strategies  that  need  to  be  implemented.  Such  strategies  will  ultimately  increase 
the  effectiveness  of  course  materials  and  content  delivery.  Revisions  that  take  place 
during  the  formative  evaluation  process  can  be  implemented  prior  to  course 
completion. 

Summative  evaluation  will  occur  at  the  conclusion  of  the  course.  It  will  provide  data 
on  the  effectiveness  of  the  course  and  may  include  data  collected  during  formative 
evaluations.  Data  gathered  through  the  summative  evaluation  process  will  help  in 
planning  future  courses.  Summative  evaluation  will  often  include  pre-testing,  post- 
testing,  and  the  gathering  of  other  information  that  is  obtained  from  tests  and 
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questionnaires.  Other  information  can  also  be  collected  that  provide*)  information 
such  as  persona]  profiles  of  participating  students. 

The  development  of  a  course  assessment  strategy  is  tho  key  to  determining  what 
changes  need  to  be  made  to  increase  course  effectiveness  and  efficiency.  The  Alaska 
University  (1989b)  suggest  considering  the  evaluation  of  the  following: 

*  Audience  attitude  and  performance. 

*  Strengths  and  weaknesses  of  program. 

*  Effectiveness  of  teaching  techniques  and  delivery  systems. 

*  Instructor  effectiveness. 

*  Degree  to  which  course  objectives  were  met.(p.  6) 

Particular  instruments  can  be  developed  and  administered  to  distant  and  home  site 
learners.  Pre-test  and  post-test  instruments  will  obviously  determine  if 
instructional  delivery  was  effective  in  producing  the  desired  learning  outcomes  and 
will  be  important  to  the  summative  evaluation  process. 

Questionnaires  and  tests  can  be  developed  to  perform  both  fornintive  and 
summative  evaluation.  It  is  important  to  remember  that  evaluation  instruments 
have  already  been  developed.  Check  to  see  if  questions  can  be  applied  to  the 
particulars  of  the  course  to  evaluate. 

Biner  (1993)  provides  a  four-step  process  for  the  development  of  an  instrument  to 
measure  student  attitudes  toward  televised  courses.  The  steps  include: 

1  •  Generating  items  related  to  course  satisfaction. 

a.  Instruction/Instructor  Aspects 

b.  Technological  Aspects 

c.  Course  Management/Coordination  Aspects 

2  -  Defining  dimensions  underlying  items. 

3  -  Selecting  content  valid  items. 

4  •  Writing  and  pre-testing  the  instrument. 

Harrison  et  al  (1991)  determined  that  the  three  most  consistently  mentioned 
components  of  distance  education  programs  were  instruction,  management,  and 
logistics.  Example  elements  associated  with  each  of  these  components  include: 

Instruction 

*  instructional  strategies  such  as  the  pacing  of  instruction,  the  use  of 
work-related  examples,  the  formation  of  study  or  support  groups 
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*  effectiveness  of  instructional  materials 


*  the  methods  used  for  providing  feedback,  counseling,  or 
motivation  of  learners 

Management 

*  organizational  structure 

*  communications  between  the  program  and  clients, 
instructors,  or  students 

*  how  available  resources  are  used 

*  the  policies  and  procedures  of  the  distance  education  program. 
Logistics 

*  the  quality  of  the  video 

*  the  on-time  delivery  of  instructional  programs 

*  an  efficient  method  of  providing  off-air,  course-related 
materials 

*  the  instructional  environment  at  the  receiving  site(s) 

Such  elements  can  he  included  in  the  assessment  instruments  used  in  formative 
and  summativo  evaluations.  Appendix  E  contains  several  sample  evaluation 
instruments  that  can  be  utilized  to  perform  evaluations.  Because  assessment  is  so 
important  in  providing  information  needed  to  determine  revision  needs,  such 
instruments  should  be  developed  during  the  course  design  and  planning  stage  of  the 
systematic  planning  process  to  ensure  that  assessment  is  effective  and 
implemented  on  a  timely  schedule. 

An  Instructional  Design  Model  for  Distance  Education  Via  I  TV 

Based  on  the  literature  and  our  own  experience  with  distance  education  via 
interactive  television,  we  suggest  the  following  nine  step  model  for  designers  of 
college  level  distance  education  courses: 

1.  Identify  Course  Goals 

2.  Analyze  and  Organize  Content 

3.  Write  Performance  Objectives 

4.  Identify  Learner  Characteristics 

5.  Develop  Lesson  Plans 

6.  Develop  and  Select  Instructional  Materials 

7.  Design  and  Conduct  Formative  Evaluation 

8.  Modify  Instructional  Plan 

9.  Design  and  Conduct  Summative  Evaluation 
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This  model  is  an  adaptation  of  the  Dick  and  Carey  model  discussed  previously  and 
is  based  on  the  unique  characteristics  of  distance  learning  via  ITV,  and  our  own 
research.   We  think  that  it  is  well  suited  to  the  delivery  of  graduate  level  curses 
and  takes  into  consideration  the  mature  learner.  This  model  has  evolved  through 
experience.   It  has  been  used  successfully  at  Texas  Tech  for  several  years  and  we 
therefore  offer  it  for  your  consideration.  The  model  is  implemented  as  follows: 

To  be  practical  in  terms  of  planning  and  development  time  and  cost  effectiveness, 
distance  education  courses  must  be  planned  and  conducted  with  resources  similar 
to  what  is  afforded  traditional  campus-based  courses.    First,  a  planning  team 
should  be  established.  Members  of  planning  teams  frequently  include  content 
experts,  instructional  designers,  video  production  experts,  graphic  artists,  media 
specialists,  and  others.   However,  as  a  practical  matter,  limited  college  and 
university  budgets  and  resources  usually  limit  the  membership  of  the  planning 
team.   We  suggest  that  planning  team  consisting  of  the  teacher,  an  instructional 
technologist,  and  a  student  or  potential  student  of  the  course.   It  is  also  possible 
that  an  experienced  ITV  teacher  may  constitute  the  entire  planning  team. 

1.  Identify  Instructional  Goals. 

Goals  are  the  broad  statements  of  expected  outcomes  for  the  course.  Identification 
of  the  instructional  goals  are  primarily  the  responsibility  of  the  course  instructor.  If 
the  course  is  designed  to  meet  a  specific  purpose  (eg.  certification  or  degree 
requirements)  the  basic  content  of  the  course  may  be  predetermined.   The  goal 
statements  will  therefore  reflect  this.  Usually,  courses  taught  via  ITV  arc1  also 
taught  in  traditional  campus  classrooms.  The  goals  of  the  ITV  course  should  hv 
basically  the  same  as  campus-based  courses.  We  have  offered  an  introductory 
course  in  computer  applications  for  educators  both  on  campus  and  via  ITV.  Sonu* 
sample  goals  for  this  course  are  as  follows: 

1 .  Demonstrate  an  understanding  of  basic  computer  terminology,  hardware 
and  software  both  as  a  classroom  and  personal  productivity  tool. 

2.  Kvaluate,  select  and  recommend  educational  software  and  prepare 
appropriate  instructional  design  in  order  to  integrate  the  use  of  small 
computers  into  specific  content  areas. 

3.  Explore  developmental  issues  related  to  the  creation  of  appropriate 
instructional  software. 


These  goals  appear  on  the  course  syllabus  for  both  the  regular  on-campus  classes 
and  ITV  classes. 

2.  Conduct  Content  Analysis 

The  teacher  is  a  content  specialist  and  therefore  will  be  knowledgeable  concerning 
the  content  of  the  course.   Texts,  accreditation  association  guidelines,  and  state 
curriculum  guides  may  also  be  helpful.  We  divide  the  course  into  logical  segments 
representing  one  class  each  and  begin  to  develop  a  lesson  plan  for  each  class.  The 
topics  and  content  outline  for  each  class  are  specified  First.   The  instructional  time 
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for  any  college  course  must  meet  the  minimum  requirements  of  the  college/university 
and  the  accreditation  agency,  usually  35-40  hrs  each.   However,  allocation  of  class 
time  may  be  different  for  ITV  course  than  for  on-campus  courses.  Some  courses  will 
require  guided  practice  at  remote  sites,  which  are  not  part  of  televised  lessons. 
This  may  be  scheduled  as  a  laboratory  or  may  be  conducted  as  part  of  regular 
instructional  time. 

3.  Write  Performance  Objectives 

Measurable  objectives  are  written  for  each  lesson.   These  are  taken  from  the  goal 
statements  and  are  stated  in  terms  of  observable  student  behaviors.  Measurable 
objectives  help  clarify  exactly  what  is  expected  in  the  minds  of  both  the  teachers  and 
the  learners.  Techniques  of  stating  performance  objectives  are  beyond  the  acope  of 
this  paper.  A  sample  set  of  objectives  for  a  lesson  on  integrated  software  for  our 
educational  computing  course  follows  is  given  here  as  a  sample: 

Topic:  Introduction  to  Integrated  Software 
Objectives: 

Students  will  be  able  to: 

1.  Define  integrated  software. 

2.  List  several  examples  of  integrated  software  packages 

3.  List  examples  of  computer  applications  which  an1  frequently  included  in 
integrated  software. 

4.  List  examples  of  products  which  mny  be  produced  with  integrated  soft  ware, 

5.  Discuss  the  relative  advantages  and  limitations  of  integrated  software  packages 
as  compared  with  stand-alone  software. 

If  the  teacher  is  not  an  instructional  designer,  the  lei'hnulogiKt  Jeuihl  uhnImI  I  he 
course  instructor  in  stating  performance  objective*  for  em  h  rlm^n    The  htudent 
member  of  the  planning  team  can  be  asked  to  review  the  evolving  Ipmhoii  plana  hir 
suggestions.   Clarity  of  goals  and  object  ive  statements-,  terininnloiiv  lined,  and 
suggestions  on  content  to  be  inch)'  ed  can  he  reviewed 

4.  Identify  Learner  Characteristics 

There  are  many  ways  to  identify  learner  rhnraeterialirrt    We  §•  m  1 1  it  - 1  mhn  unit  inn  ill 
the  first  class  meeting.  This  begins  with  intrndui'tnuia  anil  ruinplelliin  nl  a  hiiet 
student  profile  form.   This  includes  such  iiilni  ninlinn  ait  ahuli'id'a  nuiini  i  nilli-e  nl 
study,  reason  for  taking  the  course,  previous  expiM  lew  v  ie|evunl  In  I  he  nniine,  mid 
place  of  employment.  We  also  use  both  the  ('aullcM  I  teaming  Ml  vleu  Inviutui  v  it  ml 
a  teacher-made  pre  test  which  is  given  at  the  hint  i'Iiihh  meeting    The  t  'airfield, 
which  was  discussed  earlier,  provides  information  on  hew  the  aluileiil  leainn  heal 
and  suggests  the  kinds  of  activities  whieh  may  he  liel|ilnl  to  hnn  in  her    The  pre 
test  provides  information  about  how  much  the  tit mlent  already  knows  nlmiii  the 
course  goals  and  objectives.   Since  we  deal  with  mature  leniueia,  weolten  Uhk  ha 
self-assessment.    The  information  from  these  mall  amenta  in  |uumptly  neat  tut  he 
instructor  by  FAX  or  data  conimunicalioan    The  data  im  iintily/eil  ami  detail  in 
added  to  the  lesson  plans  based  on  the  information.    |»'oi  eKiimple,  it  moat  nt  the 
students  arc  found  to  he  field-independent  learners,  mure  arllvild'H  whirl)  provide 
for  independent  study  are  inrhided  in  the  murai*    Student  profile  ranis  are  hIhii 
made  which  are  referred  to  by  the  instructor  as  t  he  course  progresses     We  use 


these  to  try  to  make  assignments  relevant  to  the  student's  interests,  level  of 
knowledge,  and  learning  style. 

5.  Develop  Lesson  Plans 

The  basic  instructional  plan  is  the  syllabus  which  is  prepared  first.   However,  the 
details  of  the  instructional  plan  are  contained  in  the  lesson  plans.   The  framework 
of  each  lesson  plan  was  established  in  the  content  analysis  and  writing  of 
performance  objectives  for  each  lesson.   Now  it  is  time  to  add  detail  to  each  lesson. 
With  knowledge  of  the  learners,  the  objectives  and  topics  can  be  adjusted. 
Relevant  examples  can  be  developed.  Activities  can  be  planned.  Requirements  for 
media  and  materials  can  be  determined.   The  technologist  works  with  the  teacher 
to  plan  activities  which  work  best  with  the  ITV  medium.  Lesson  plans  will  be  fully 
developed  as  the  course  is  conducted.   However,  it  is  important  to  have  long  range 
plans.   In  addition  to  the  basic  need  for  good  planning,  the  ITV  staff  will  need  to 
know  what  to  expect  and  production  of  materials  must  be  done  in  advance.  We  try 
to  have  final  plans  ready  at  least  two  weeks  in  advance  of  each  class. 

6.  Develop  and  Select  Instructional  Materials 

Earlier  it  was  pointed  out  that  ITV  is  an  inherently  visual  medium.   If  a 
traditional  lecture  is  poor  technique  in  a  regular  class,  it  is  deadly  on  ITV.  Visuals, 
video  demonstrations,  and  commercial  media  can  be  used  to  present  the  content  in 
a  manner  which  is  more  likely  to  maintain  the  students  attention  and  help  thorn 
understand  and  retain  the  information.  Specifications  for  the  media  are  developed 
in  the  instructional  plan.   Frequently,  the  teacher  produces  the  media  or  the 
campus  media  service,  ITV  staff,  or  students  may  help. 

7.  Design  and  Conduct  Formative  Evaluation 

As  the  course  progresses,  frequent  feedback  is  obtained  both  informally  and 
formally.   Question  and  answer  sessions  are  held  during  each  class.   Brief  quizzes 
are  given  and  student  assignments  are  collected  regularly.   We  specifically  try  to 
determine  which  methods  and  materials  are  preferred  and  to  what  degree  objectives 
are  being  mastered. 

8.  Modify  Instructional  Plan 

Modifications  to  the  lesson  plans  are  made  as  the  course  progresses  based  oq 
feedback  from  the  students.   The  student  member  of  the  planning  team  is  also 
frequently  used  as  a  sounding  board  and  to  make  suggestions.    Material  that  was 
not  mastered  may  be  clarified  by  presenting  it  in  another  manner.   Sometimes,  new 
timely  topics  are  added. 


9.  Design  and  Conduct  Summative  Evaluation 

At  the  conclusion  of  the  course,  a  post-test  and  a  course  evaluation  are 
administered.   The  post-test  is  either  the  same  as  the  p^e-test  or  a  more  detailed 
version  of  it  and  provides  detailed  information  concerning  how  well  each  objective 
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was  mastered.   The  course  evaluation  that  we  use  is  a  standard  course  evaluation 
used  at  Texas  Tech  to  which  we  have  added  several  items  concerning  delivery  of  the 
course  via  ITV.   This  information  is  useful  in  evaluating  the  success  of  the  course 
and  for  providing  information  for  the  next  time  the  course  is  offered.  The  course 
planning  team  reviews  and  evaluates  the  data  and  makes  recommendations  for 
future  courses. 

Summary 

Distance  education  delivery  via  ITV  is  clearly  an  effective  means  of  providing 
instruction  to  learners  at  a  distance.  Interaction  among  course  participants  can 
vary  over  a  wide  range  because  of  the  capabilities  of  this  delivery  system.  As  a 
result,  it  is  important  for  educators  who  use  this  technology  for  delivering  instruction 
to  know  how  to  use  the  system,  understand  its  capabilities,  and  know  how  to  apply 
those  capabilities  to  enhance  instructional  delivery.  This  in  turn  will  provide  the 
best  possible  learning  outcomes  for  participating  learners. 

ITV  technology  provides  many  advantages  as  an  instructional  delivery  system. 
Advantages,  such  as  the  ability  to  provide  immediate  feedback  to  learners  and  have 
interactive  personal  participation  among  all  sites,  enable  this  delivery  format  to  be 
highly  effective.  Additionally,  students  benefit  from  this  format  by  experiencing 
similar  learning  and  achievement  levels  of  students  who  receive  instruction  through 
the  traditional  classroom  setting. 

Other  advantages  of  ITV  that  play  an  important  role  in  providing  effectiveness 
include  the  visual  capabilities  of  the  system,  open-line  audio  transmission,  and  the 
sharing  of  many  resources  including  master  teachers. 

Research  indicates  that  instruction  delivered  via  ITV  requires  structured  planning 
and  development.  The  use  of  instructional  design  principles  to  achieve  effectiveness 
in  course  planning  and  instructional  delivery  is  imperative.  There  are  many 
instructional  design  models  that  can  help  the  course  designer  accomplish  the 
planning  and  development  of  DE  courses.  Although  no  one  specific  model  has  been 
determined  to  be  the  most  effective  in  aiding  the  course  designer,  the  use  of  such 
models  has  proven  to  be  effective  in  producing  interactive  television  instructional 
delivery  effectiveness. 

Course  planning  for  ITV  delivery  requires  the  designer  to  know  the  characteristics 
associated  with  distance  education  systems,  the  available  resources,  how  course 
content  is  to  be  delivered,  and  learner  characteristics  and  learning  styles.  Planning 
teams  can  be  utilized  and  the  number  of  participants  can  vary  from  one  to  many. 

Studies  show  that  DE  teachers  are  concerned  with  selecting  course  content  and 
materials,  covering  the  course  content,  class  process,  on-going  planning  activities, 
student  participation,  teacher-student  communication,  and  student  characteristics. 

McCiroal's  (1991)  guidelines  to  help  the  ordinary  teacher  produce  instructional 
materials  for  distance  education  systems  are  helpful  and  are  similar  to  those  found 
in  the  literature.  Also,  Parker  and  Monson  (cited  in  Moore  &  Thompson  et  al,  1990) 
reveal  techniques  important  to  course  development  that  include  humanizing, 
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participation,  message  style  and  feedback.  Interactive  study  guides  are  course 
materials  that  can  also  be  developed  to  accommodate  the  ITV  medium. 

Specifically  targeting  conditions  that  will  help  students  learn  in  the  DE  setting  is 
important.  Teaching  methods  and  models  will  prove  effective  with  the  ITV  medium. 
Knowing  the  characteristics  and  learning  styles  of  the  learners  will  help  increase  the 
effectiveness  of  the  instruction  development  and  delivery  processes.  Including 
delivery  strategies  that  accommodate  for  both  FI  and  FD  learners  will  provide  for  a 
wider  range  of  learners. 

Existing  research  and  literature  provide  these  recommendations  for  training  0 
teachers  about  ITV: 

*  Provide  teachers  with  an  overvie  v  of  the  technology  and  how  it  works. 

*  Provide  hands-on  guided  practice  on  the  use  of  ITV  technology. 

*  Incorporate  the  effective  elements  of  instruction  as  major  parts  of  the 
training  session. 

*  Have  periodic  follow-up  inservice  and  on-site  coaching  to  ensure  long  term 
training  benefits. 

*  Use  with  teachers  who  volunteer  for  ITV  training.  (Thompson,  Simonson 
and  Hargrave,  1992,  p.  42) 

*  Establish  the  amount  of  time  needed  to  prepare  and  teach  distance 
delivered  courses. 

*  Develop  methods  to  establish  and  maintain  effective  communication  with 
distant  students. 

*  Provide  experiences  with  other  faculty  members. 

*  Incorporate  strategies  for  adding  visual  components  to  audio  courses. 

*  Plan  and  manage  of  organizational  details  involved  in  distance  delivery. 

*  Use  strategies  to  encourage  group  cohesion  and  student  motivation. 

In  delivering  instruction  through  the  ITV  medium,  existing  research  and  literature 
provide  effective1  teaching  strategies  and  guidelines  for  personalizing  the  ITV 
classroom.  Teachers  should: 

*  Adapt  instruction  to  meet  the  varied  needs  of  the  students,  the  content, 
and  the  limitations  of  the  delivery  system. 

*  I  Iohl  a  pre  course  audio  (or  ITV)  conference  to  increase  student  familiarity 
and  comfort  with  the  system, 

*  Familiarize1  students  with  each  other  and  with  the  instructor: 
develop/distribute  student  and  instructor  biographical  sketches,  make  on-air 
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introductions,  have  students  state  their  names  and  locations  when  they 
address  the  group. 


*  Maintain  phone-in  office  hours  so  students  can  call  collect. 

*  Develop  skills  to  facilitate  students'  learning  on  their  own  and  in  concert 
with  other  students  at  a  distance. 

*  Work  with  students  to  minimize  and  rectify  technological  problems. (p.  3) 

*  Emphasize  and  encouiage  active  student  participation. 

*  Teach  to  the  camera.  This  gives  the  remote  student  the  impression  of  eye 
contact  which  is  critical  to  personalization. 

*  Travel  to,  and  teach  from,  each  remote  site  if  possible. 

*  Devise  formative  evaluation  techniques  in  order  to  assess  the  success  of 
the  class  as  it  is  being  taught.  (Thompson,  Simonson  and  Hargrave,  1992, 
p.  42) 

DE  taught  via  ITV  can  be  effective  and  rewarding.  Proper  course  design, 
development,  implementation,  and  evaluation  will  ensure  that  the  DE  learner 
receives  the  maximum  benefits  from  the  delivery  medium.  The  research  and 
literature  provide  information  to  help  the  DE  teacher  to  understand  DE  technology 
and  systems,  and  plan  and  deliver  effective  instruction.  As  a  result,  learning 
opportunities  for  DE  learners  will  continue  to  improve  as  new  strategies  and 
advancement  in  technology  enhance  the  delivery  of  instruction  via  ITV. 

REFERENCES  may  be  found  in  Appendix  E 
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This  paper  will  explore  a  conditions-based  approach  to  development  of  micro-level 
organizational  strategies  for  instruction.  The  paper  draws  upon  ideas  generated  during 
development  of  the  recently-published  text,  Instructional  Design  (Smith  and  Ragan,  1993)  as 
well  as  subsequent  work. 

Reigeluth  (1983)  presents  three  components  of  instructional  strategy:  organizational, 
delivery,  and  management  strategies.  Organizational  strategies  refer  to  content 
presentation  and  sequence  concerns,  delivery  strategies  refer  to  media  and  grouping  concerns 
and  management  strategies  refer  to  matters  of  scheduling  and  resource  allocation.  Although 
all  three  components  are  of  concern  to  instructional  designers,  organizational  strategies  are 
hoth  the  primary  concern  for  this  paper  and  are  a  component  which  has  profound  influence 
on  the  other  components  as  well.  Little  prescription  has  been  said  to  be  available  (the  "black 
box"  )  for  organizational  strategies:  how  will  information  be  presented,  what  content  will  be 
explored,  in  what  activities  will  learners  engage?  What  will  happen  during  instruction  in 
what  sequence  or  by  what  sequence-controlling  event(s)? 

Too  often  in  instructional  design  literature,  (or  in  common  perceptions  of  the  literature)  when 
the  focus  of  the  design  process  is  on  the  "develop  instructional  strategy "  prescription  is 
replaced  with  "just  do  it."  In  phases  of  the  design  process  both  before  and  after  strategy 
development,  instructional  design  technology  has  justified  the  term  "technology/*  Principles 
and  procedures  abound  to  guide  the  designer  in  performing  instructional  analysis;  the  same 
can  be  said  for  development  of  assessments  and  for  conducting  summative  and  formative 
evaluation.  But  in  strategy  development,  too  often  the  principles  which  would  guide  one  in 
making  decisions  have  been  clouded  in  mystery.  It  is  a  classic  case  of  the  "black  box"  in 
which  both  inputs  and  outputs  are  known,  but  the  process  itself  appears  to  be  unknown. 

It  is  our  contention  that  more  is  known  that  is  of  a  prescriptive  nature  regarding  the  design 
of  instructional  strategies,  specifically  organizational  strategies,  than  is  generally 
acknowledged.  The  literature  has  been  scat  red  and  difficult  to  synthesize  into  a  coherent 
whole.  And.  although  the  authors  present  an  approach  which  itself  is  a  synthesis,  it  is  not  a 
synthesis  of  everything  out  there  on  instructional  strategies,  since  thus  far  a  reconciliation  of 
all  potentially  useful  points  of  view  has  not  been  possible. 

The  approach  to  be  discussed  rests  on  what  can  be  described  as  the  "conditions  model"  of 
instruction.  The  conditions  model  reflects  the  idea  that  differences  in  learning  tasks, 
primarily  qualitative  differences  in  cognitive  processing  required  of  different  learning  tasks, 
can  suggest  different  ways  by  which  instruction  can  either  supply  the  processing  needed  or 
assist  learners  to  generate  the  needed  processing.  The  conditions  model  is  most  widely 
known  in  the  work  of  R.M.  Gagne  (1985),  but  is  also  evident  in  work  by  many  others 
including  Merrill  (1983),  Reigeluth  (1983),  Landa  (1974),  and  Tennyson  (1986). 
Although  there  are  questions  about  and  limitations  of  the  conditions  model  which  we  may 
not  be  able  answer  or  rebut  at  present  (e.g.,  concerns  from  the  constructivist  orientation 
about  the  feasibility  of  having  stable  enough  i,objectives,,  for  learning  to  analysis  and 
subsequent  instruction  based  on  those  objectives),  there  is  one  important  perceived  limitation 
in  this  approach  which  we  helieve  we  have  surmounted  and  which,  in  so  doing,  greatly 
expands  the  reasonableness  and  utility  of  the  model.  That  limitation  has  been  more 
perceived  than  real,  in  our  view,  yet  it  appears  that  ours  may  be  the  first  sufficient 
explication  to  thoroughly  counter  the  argument  of  this  particular  limitation.  That  limitation 
has  been  the  notion  that  conditions-based  models  (and  by  association,  the  "instructional 
design"  approach)  prescrihe  only  supplantive  instruction  or  that  the  models  are  inherently 
biased  toward  supplantation. 
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It  is  easy  to  see  how  the  idea  that  conditions  models  are  associated  with  supplantive 
strategies,  even  though  we  believe  this  association  to  be  largely  the  result  of 
miscommunication  and  misunderstanding.  A  good  example  of  what  might  lead  a  person  to 
assume  that  the  conditions  models  suggest  supplantive  strategies  is  seen  in  the  wording  of 
the  "events  of  instruction"  (Gagne.  1972): 

1.  Gaining  attention 

2.  Informing  the  learner  of  the  objective 

3.  Stimulating  recall  of  prerequisite  learning 

4.  Presenting  stimulus  material 

5.  Providing  learning  guidance 

6.  Eliciting  performance 

7.  Providing  feedback 

8.  Assessing  performance 

9.  Enhancing  retention  and  transfer 

Gagne's  explanations  of  the  events  of  instruction  occasionally  suggest  that  he  may  have 
intended  that  the  events  could  at  times  be  other  than  "instruction-supplied"  even  though 
such  suggestion  is  not  present  in  the  way  the  events  are  worded. 

Regardless  of  Gagne's  original  intent,  it  is  easy  to  view  events  of  instruction  from  a  position 
which  is  neutral  with  regard  to  the  supplantive/generative  question,  placing  the  selection  of 
strategy  in  that  regard  an  element  in  the  design  process. 

In  the  present  author's  recent  text  (Smith  and  Ragan,  1993),  a  revision  of  the  events  of 
instruction,  which  we  label  "expanded  events  of  instruction,"  is  presented  in  which  both 
generative  and  supplantive  interpretations  are  suggesting  by  the  wording  of  each  event. 


Supplantive  Version 

Generative  Version 

Introduction 

1. 

Gain  attention  to  lesson 

Activate  attention 

2. 

Inform  learner  of  purpose 

Establish  purpose 

3. 

Stimulate  learner's  attention 

Arouse  interest  &  motivation 

4. 

Provide  overview 

Preview  lesson 

Body 

5. 

Stimulate  recall 

Recall  prior  knowledge 

6. 

Present  information 

Process  information 

7. 

Gain  &  focus  attention 

Focus  attention 

8. 

Guide  or  suggest  use  of  lng.  strats. 

Employ  learning  strategies 

9. 

Elicit  response 

Engage  in  practice 

10. 

Provide  feedback 

Evaluate  feedback 

Conclusion 

11. 

Provide  summary  &  review 

Generate  summary  &  review 

12. 

Enhance  transfer 

Transfer  learning 

13. 

Provide  remotivation  &  closure 

Remotivate  and  finish 

Assessment 

14. 

Conduct  assessment 

Assess  performance 

16. 

Provide  feedback  and  remediation 

Evaluate  feedback  and  seek 

remediation 

Kifjure  1 :  Expanded  Events  of  Instruction  (adapted  from  Smith  &  Ragan,  1993) 
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We  suggest  that  choice  of  instructional  strategy  form-from  relatively  supplantive  to 
relatively  generative-should  be  made  on  the  basis  of  consideration  of  certain  critical  • 
characteristics  of  the  learning  task,  learners,  and  context  for  learning,  specifically:  time 
available  for  instruction,  learner  aptitudes,  motivation,  prior  knowledge,  availability  of 
cognitive  strategies,  criticality  of  task,  and  future  learning  requirements.  Thus,  if  time, 
learner  characteristics,  and  so  forth  will  allow  it,  the  summarizing  event,  for  example,  may 
be  generated  by  the  learners  rather  than  being  supplied  by  instruction.  A  guideline  we 
would  propose  is:  make  the  instruction  as  generative  as  conditions  allow.  Here  we  follow  a 
different  path  from  those  who  recommend  always  employing  a  generative  (exploratory, 
inquiry,  inductive)  strategy,  and  we  take  a  different  view  from  those  who  would  always 
employ  a  supplant  ive  one.  A  more  extensive  treatment  of  the  generative-supplantive 
strategy  dialogue  can  be  seen  in  Smith  (1985). 

The  neutrality  of  instructional  events  is  maintained  in  the  interest  of  reducing  designers' 
assumptions  or  predispositions  for  one  general  form  of  instruction  or  another.  Application  of 
the  expanded  events  should  serve  to  reduce  the  frequency  and  validity  of  the  criticism  that 
instructional  designers  tend  to  develop  a  supplantive  form  of  instruction,  neglecting 
approaches  which  can  sometimes  be  more  desirable. 

The  second  aspect  of  the  expanded  events  concept  is  application  to  different  types  of  learning. 
In  the  following  we  will  describe  strategy  recommendations  for  six  major  types  of  learning: 
declarative  knowledge,  concept  learning,  procedural  rule  learning,  relational  rule  learning, 
problem  solving,  attitude  learning,  and  the  learning  of  psychomotor  skills. 
Recommendations  which  follow  have  been  pulled  together  from  many  sources.  First  it  should 
be  acknowledged  that  the  base  of  work  by  Gagne  (1985)  and  Gagne  and  Briggs  (1979) 
provided  a  point  for  expansion.  Many  of  the  following  summary  frames  appear  in  Smith  & 
Ragan  (1993);  some  were  developed  for  this  paper.  A  few  key  sources  are  noted  for  each 
discussion  and  in  the  references  section  of  this  paper,  and  full  references  are  provided  in 
Smith  and  Ragan  (1993). 
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Strategy  Elements  for  Declarative  Knowledge  Learning 


Introduction 

Deploy  attention 

Arouse  interest  and  motivation 

•  Use  of  novel,  conflictual  and  paradoxical  events,  the 
interjection  of  personal/emotional  elements,  and  making 
clear  how  the  present  learning  relates  to  other  learning 
tasks 

Establish  instructional  purpose 

*  Relate  instr.  goals  to  personal  goals  or  job  reqmnts.,  make 
instrl.  goals  relevant,  present  goal  in  interesting  format, 
remind  lnrs.  of  relevant  lng.  strats.,  point  out  requirements 
for  successful  attainment  of  the  obj.,  &  let  lnrs.  know  the 
form  in  which  they  need  to  remember. 

Preview  lesson 

•  Advance  organizers  or  epitome  can  be  useful  form  of 
preview,  also  outlines  or  maps. 

Body 

Recall  prior  knowledge 

•  Advance  organizers,  use  of  metaphoric  devices,  and 
reviews  of  prerequisite  concepts 

Pvnrpcc  i  n  ttii*iti  51  ti  on 
r  xutcos  iiiLVJi  iLLa  null 

organization:  clustering  and  chunking 
elaboration:  elaboration  into  sentences 

•  Facts/lists 
assoc  use  of  images 

org:  expository  and  narrative  structures,  recognizing 
patterns,  clustering  and  chunking,  and  elaboration. 

•  Organized  discourse: 

assoc:  imagery,  metaphoric  devices 

org:  analysis  of  expository  and  narrative  structures,  use  of 
graphic  organizers-  frames,  concept  mapping 
elaboration:  elaboration  model 

Focus  attention 

Underlining,  listing,  &  reflecting;  Questions:  pre-  &  post-, 
embedded 

Employ  learning  strategies 

•  Previously  noted  strategies  (all  but  advance  organizer) 

•  Mnemonic  techniques  such  as  single  use  coding,  pegwords, 
the  method  of  loci,  keywords,  and  the  use  of  rhymes, 
stories,  or  jingles. 

•  Rehearsal 

Practice 

•  Role  of  practice,  consider  diff.  needs  for  pract.  for  recall  vs. 
recognition  lng.  tasks  and  for  verbatim  vs.  paraphrased 
recall,  consider  needs  for  spaced  practice,  &  the  role  of 
automaticity  in  declarative  knowledge 

Evaluate  feedback 

•  Consider  feedback  needed  for  labels,  facts,  and  lists  (eval. 
correctness  of  associations  of  elements)  as  contrasted  with 
the  feedback  needed  for  organized  discourse 
("understanding") 

Conclusion 

Summarize  and  review 

•  Tuning  cognitive  structures,  learner-generated 
summaries,  interim  summaries 

669 


ICG 


Transfer  knowledge 

•  Increase  the  number  of  possible  connections  in  the 
learners  mental  map,  the  role  of  application  in  a  variety  of 
settings,  learners  inference- making 

Remotivate  and  close 

•  Show  how  learning  can  help  student. 

Assessment 

Assess  performance 

•  Care  required  to  b  congruent  with  objective 

Feedback  and  remediation 

•  Identify  and  clarify  needs  for  learning 

Propositional  networks:  Anderson,  1976;  Schema:  Minsky,  1975;  Rummelhart  &  Ortony,  1977; 
Cognitive  process  of  learning:  E.  Gagne,  1985;  Illustrations  in  lng.  org.  discourse:  Duchastel,  1978; 
Narrative  structures  &  lng.  org.  discourse:  Armbruster  &  Anderson,  1985;  Graphic  organizers: 
Holley  &  Dansereau,  1984;  Focusing  effect  of  questions:  Bull,  1973;  Attentional  effect  of  questions: 
Schramm,  1964;  Mnemonic  techniques:  Atkinson,  1975,  Pressley,  Levin,  &  Delaney,  1982; 
Generative  summaries,  Hidi,  1985 
Strategy  Elements  for  Concept  Learning 


Introduction 

Deploy  attention 

Arouse  interest  and  motivation 

•  Highlight  concept  label,  use  unusual  picture  or  humorous 
story  regarding  concept,  provide  interesting  information 
on  origin  or  history  of  concept,  and  present  first  matched 
example  and  nonexample.  Use  inquiry  approach. 

Establish  instructional  purpose 

•  State  explicitly  in  expository  lesson.  Delay  statement  in 
inquiry  lesson. 

Preview  lesson 

•  Overview  process  of  inquiry  approach.  Point  out 
importance  of  examples  and  nonexamples  and  practice  in 
lesson. 

Body 

Recall  prior  knowledge 

•  Review  concepts  constituting  critical  attributes  of  concept. 
Use  techniques  such  as  informal  questioning,  formal 
pretest,  advance  organizer,  or  analogy 

Process  information 

•  Expose  to  best  example  and/or  definition.  Emphasize 
criterial  attributes.  Consider  matched  examples  and 
nonexamples.  Present  concept  in  range  of  settings  with 
diversity  of  non  relevant  attributes. 

Focus  attention 

•  Isolate  criterial  attributes  in  examples  with  highlighting 
such  as  boldface  type,  color,  or  a  simplified  drawing. 

Employ  learning  strategies 

•  Generate  concept  maps,  analogies,  mnemonics  or  images. 

Practice 

•  Identify  examples  from  previously  unencountered 
instances,  which  range  in  difficulty  and  settings.  Explain 
categorizations.  Generate  examples. 

Evaluate  feedback 

•  Feedback  contains  attribute  isolation. 

Conclusion 

Summarize  and  review 

•  Restate  criterial  attributes. 

•  Repeat  or  paraphrase  key  information. 

Transfer  knowledge 

•  Apply  outside  classroom. 

•  Provide  further  examples. 
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Reniotivate  and  rlosu 

•  Show  how  learning  can  help  student. 

Akhohk  performance! 

•  Test  ability  to  isolate  criterial  attributes  in  examples  and 
point  out  their  absence  in  nonexamples. 

•  Test  including  range  of  common  and  non  relevant 
attributes. 

I'VctlluH'k  ami  remediation 

•  Provide  score  or  other  performance  summary. 

•  Identify  problems  of  over-  and  under-  generalization. 

Conrepl  learning:  Klausmoier,  1980,  Merrill  &  Tennyson,  1977;  Wilson,  1987;  Strategies  for 
leaching  concepts:  Tennyson  &  Cocchiarella,  1986;  Tessmer,  Wilson  &  Driscoll,  1990;  Use  of 
examples:  Ali,  Use  of  analogies:  Newby  &  Stepich,  1987;  Concreteness  of  illustrations: 

Smith  &  Smith,  IHHI 


Strategy  Elements  for  Kelational  Rule  Learning 


Introduction 

— 7\ — i  '■  

Deploy  atteut ion 

— — : — :  r — : — : — ~i — ui 

•  (>unosity-evoking  situation/problem 

Establish  instructional  purpose 

•  Understand/apply  principle,  relationship  between 
concepts 

Arouse  interest  and  motivation 

•  (Curiosity-evoking  situation 

Preview  lesson 

•  Inquiry=directions;  expository=outline 

Rnrlv 
Duuy 

Recall  prior  knowledge 

•  Review  component  concepts 

Process  information 

•  Present/induct  relationship,  state  in  principle  form, 
demonstrate  application 

Focus  attention 

•  Direction  &  size  in  change  of  one  variable  when  other 
variable(s)  changes 

Employ  learning  strategies 

•  Mnemonic  rule  statement,  diagram  of  relationship 

Practice 

•  Predict,  explain,  control  changes  in  concept(s)  based  on 
change  of  another;  recognize  situations  where  rule 
applies;  determine  whether  rule  correctly  applied 

Evaluate  feedback 

•  Information  on  whether  rule  applicable,  outcome  of 
application 

Conclusion 

Summarize  and  review 

•  Change  in  symbol  system;  restate  principle 

Transfer  knowledge 

•  Point  out  how  princ.  will  be  incorporated  into  prob. 
solving;  identify  life  situations 

Remotivate  and  close 

•  Relevance  to  daily  lives  or  current  probs. 

Assessment 

Assess  performance 

•  Recognize  if  principle  applicable,  apply  principle  to 
predict,  explain,  control 

Feedback  and  remediation 

•  Identify  misconceptions,  over-  or  under-  generalization 

6,1  r,j3 
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Relational  rule  learning:  Oagne,  1985;  Anderson,  (1985);  Strategies  for  teaching  relational 
rules:  Tennyson  &  Tennyson,  1975;  Joyce  &  Weil,  1986 


Strategy  Elements  for  Procedural  Rule  Learning 


Introduction 

Deploy  attention 

•  Ask  question,  demonstrate  procedure,  describe  efficiency 

Establish  instructional  purpose 

•  Describe  procedure  to  be  learned  and  range  of 

$\  Tvnlipnhi  li  t  v 

Arniicc  1  yi  t oroct  qiiH  ty> i trcj f  ion 
\J  U&C  llltcJlt.ot  dllU  Illiltl  V  (X  t 1UI 1 

•   t*  mnnQci7fi  pfTinonr^/    St  vol lOfMlltir  of  nrnrai^nro 

Preview  lesson 

•  Preview  procedure  .in  chunks 

Dooy 

Recall  prior  knowledge 

•  Review  component  comcepts,  sub  procedures,  or  related 
principle 

Process  information 

*  Simplify  complex  procedures,  situations  that  require 
proced.,  steps  in  procedure  order  of  steps,  how  to  eval. 
corr.  oi  appiic.   jviay  eiau.  over  several  iterations 

rocus  attention 

oriticai  cnar  s  oi  situations  requiring  procedure,  Key  cues 
to  transitioning  between  steps,  keywords  for  each  step, 

nip<5  for  porrppt  pomnlpfiori  of  nroppHurp 

Employ  learning  strategies 

•  Job  aid,  mnemonic  for  order  of  steps 

luentiiy  situations  requiring  proceaure,  oraer  oi  steps, 
completion  of  steps,  correct  completion  of  procedure 

Evaluate  feedback 

•  Correct  answer  w/  explan.,  checklist  or  rating  scale,  video 

IccUOalK 

C!orif*lu  %inn 

Summarize  and  review 

•  Major  steps  in  proced.,  rel.  to  principle,  appropriate 
situations  for  application 

Transfer  knowledge 

•  To  prob.  solving,  more  complex  proced's. 

Remotivate  and  close 

•  Emphasize  utility  of  proced.  in  terms  of  reliability  and 
efficiency 

Assessment 

Assess  performance 

•  Identify  situation  to  which  procedure  applies,  correct 
order  and  completion  of  steps,  recognition  of  correctly 
completed  procedure 

Feedback  and  remediation 

•  Identify  common  errors  and  misconceptions 

Procedural  rule  learning:  Gagne,  1985;  Anderson,  1985;  Teaching  procedures:  Gilbert,  1978,  Landa, 
1974;  Marcone  &  Reigeluth,  1988;  Schmidt  &  Gerlach,  1990;  Wilson,  1985. 
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Strategy  Elements  for  Problem-Solving  Learning 


Introduction 

Deploy  attention 

Arouse  interest  and  motivation 

*  Present  a  challenging  and  interesting  problem  that  is 
represented  in  a  novel  manner. 

Establish  instructional  purpose 

•  State  class  of  problem  that  learners  will  learn  to  solve. 

•  Delay  statement  in  inquiry  lesson. 

Preview  lesson 

•  Point  out  that  problems  will  become  increasingly 
complex  throughout  lesson. 

Body 

Recall  prior  knowledge 

•  Explicitly  review  relevant  prior  knowledge:  rules, 
declarative  knowledge,  or  strategies. 

•  Suggest  ways  that  learners  to  reorganize  knowledge  in  a 
more  conducive  form. 

•  Attend  to  similarities  and  differences  to  other  problem-solv 
learning. 

ng 

Process  information 

•  Encounter  simplified,  prototypical  versions  of  problem  first 

•  Verbalize  task  requirements. 

•  Provide  model  think-alouds. 

•  Decompose  problem  into  subgoals. 

Focus  attention 

•  Isolate  critical  attributes  in  given  state  and  goal  state. 

Employ  learning  strategies 

•  Generate  networks,  analogies. 

•  Monitor  success  of  solutions. 

•  Ask  guiding  questions  and  provide  hints. 

•  Represent  problem  in  alternate  forms. 

•  Use  print  or  other  media  as  a  form  of  external  storage. 

Practice 

•  Practice  identifying  and  clarifying  given  and  goal  states. 

•  Practice  decomposing  problem. 

•  Practice  evaluating  adequacy  of  a  provided  solution. 

•  Practice  with  well-defined  problems  first. 

Evaluate  feedback 

•  Model  solution  of  process  or  provide  models  of  solution. 

•  Given  hints  or  ask  questions. 

•  Provide  information  on  efficiency  as  well  as  effectiveness 

f\i  cnliifinn 

Conclusion 

Summarize  and  review 

•  Restate  criterial  attributes  of  problem  class. 

•  Summarize  effective  strategies. 

•  Suggest  ways  of  organizing  knowledge  for  storage  and  retri 

isval. 

Transfer  knowledge 

*  Find  similar  problems  outside  classroom. 

•  Explicitly  state  when  strategies  may  transfer  to  other  prob 
types. 

em 

Remotivate  and  close 

•  Review  the  importance  and  breadth  of  what  has  been  learn 

3d. 

Assessment 

Assess  performance 

•  Test  ability  to  solve  similar,  but  novel  problems,  both 
well-defined  and  poorly  defined. 

•  Test  ability  to  isolate  criterial  attributes  goal  and  given  sti 

•  Test  ability  to  evaluate  others'  solutions. 

•  Test  ability  to  justify  solutions. 

tes. 
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Feedback  and  remediation 


•  Identify  whether  problems  are  in  pattern  recognition, 
decomposition,  explaining  solution,  etc.  


Domain-specific  problem  solving:  Dunker,  1945;  Gagne,  1980,  1985;  Anderson,  1985;  de  Jong  & 
Ferguson- Hessler,  1986,  Alexander  &  Judy,  1988;  Newell  &  Simon,  1972;  Mental  models: 
Johnson-Laird,  1983;  Teaching  of  problem  solving:  Derry,  Hawkes,  &  Tsai,  1987,  Glaser,  1989; 
Mayer,  1985;  Foshay,  1991 

Strategy  Elements  for  Cognitive  Strategy  Learning 


Introduction 

Deploy  attention 

Arouse  interest  and  motivation 

Establish  instructional  purpose 

EixperieiiLt?  tdSK  wmcn  requires  ine  strategy 
#  Discuss  role  of  strategic  thinking  in  learning 

Preview  lesson 

•  Demonstrate  entire  strategy  model 

Body 

Recall  prior  knowledge 

•  Recall  previously  learned  strategies  or  tasks  which 
seem  similar 

Process  information 

•  Experience  situations  for  which  application  of  the 
strategy  is  appropriate  and  inappropriate 

•  Model  demo,  strategy  with  think-aloud 

Focus  attention 

•  Critical  attributes  of  tasks  to  which  strategy  is  approp. 

•  Cues  that  indicate  successful  application  of  strat. 

Employ  learning  si  rulegies 

•  Thinking  aloud  about  cognition  and  monitoring 
effects  of  the  strategy 

Practice 

•  Identify  contexts/tasks  to  which  strategy  is 
appropriate  and  explain  why 

•  Apply  strategy  to  increasingly  difficult  tasks 

•  Reciprocal  practice 

Evaluate  feedback 

1 

•  Peer  evaluation 

•  Croup  feedback  -  model  appropriate  application, 
examine  artifacts  of  strategy  use 

Conclusion 

Summarize  and  review 

•  Summarize  steps  &  review  tasks  to  which 
st  rategy  is  appropriate 

Transfer  knowledge 

•  Move  from  detached  to  embedded  with  prompts, 
withdraw  prompts 

•  Compare  strat.  to  others  learned  later 

Kcmotivnte  and  close 

•  Importance  of  effort  coupled  with  strategy  use 

Assessment 

Asse  s  performance 

•  Directly  observe 

•  Examine  artifacts  of  strategy  use 
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Feedback  and  remediation 

•  Was  appropriate  strategy  selected? 

•  Was  strategy  applied  correctly? 

•  Was  success  of  strategy  monitored  and 

"fix  up"  strategies  employed 

Cognitive  strategies:  Derry  &  Murphy,  1986;  Weinstein,  1982;  Gagne  &  Driscoll,  1988; 
Davidson,  1988;  Events  for  cognitive  strategy  instruction:  Davidson  &  Smith,  1990;  Deshler, 
Alley,  Warner,  &  Schumaker,  1981;  E.  Gagne,  1985;  Meichenbaum,  1977;  Pressley,  Snyder, 
&  Gargilia-Bull,  1987;  Weinstein,  1981 


Strategy  Elements  for  Attitude  Learning 


Introduction 

Deploy  attention 

ArnucP  infprPQt  j*nH  mntivjatinn 

iuUUSC  IlllCI  C3L  CI  11  LI  1 1  l\J  LI  V  CI  VILf  1 1 

•  Engaging  situation 

•  Tdpnt ifiratinn  \*/itH  pHjiTflrtprQ  nr  Qitiifltinn 
luciiimtaLiuiJ  Willi  tiiaiaLLci  9  ui  DitLicttiuii 

Establish  instructional  purpose 

•  May  be  direct  or  indirect  but  to  withhold  for  whole  lesson 

id  iiiaiiipuictLlVc 

Preview  lesson 

•  May  be  indirect  or  withheld 

T> -~ -J  __ 

Doay 

Recall  prior  knowledge 

•  Present  persuasion  before  expression  of  old  attitude 

Process  information 

•  Persuasion,  discussion,  role-play,  simulation 

Focus  attention 

use  oi  respecieu  role  niouoi  u  persuasive  tec  n.  UhCti 
•  role  model  seen  to  receive  valued  reward 

K.mnlnv  Iparni  nc  stratocripis 

•  I  f c.p  nf  Arronvtritt  mnhinnmr'S  ^lnirniiK  fnr 

cognitive  component  as  appropriate 

Practice 

•  Practice  cognitive,  behavioral,  &  affective 
-•know  what  to  do,  do  it,  know  how  it  feels 

Evaluate  feedback 

•  Emphasize  natural  consequences 

•  Include  cognitive,  behavioral,  &  affective  aspects 

Conclusion 

Summarize  and  review 

•  Clear  restatement  of  desired  behavior 
--purpose  of  instruction  should  be  clear 

Transfer  knowledge 

•  Discuss  applications,  situations 

•  Role  play,  simulations 

Remotivate  and  close 

•  Realize  how  new  learning  can  be  used 

Assessment 

Assess  performance 

•  Ideal:  behavior  in  actual  free  choice  situation 

•  Practical:  role  play  or  simulation  of  situations 

Feedback  and  remediation 

•  Emphasize  natural  consequences 

•  Include  cognitive,  behavioral,  &  affective  aspects 

Attitude  learning;  Martin  &  Briggs,  1986;  Brandhorst,  1978;  Zimbardo  &  Leippe,  1991; 
Attitude  instruction:  Fleming  &  1-evie,  1978;  Kiesler,  Collins  &  Miller,  1969;  Martin  & 
Briggs,  198H. 


Strategy  Elements  in  Psychomotor  Skill  Learning 


Introduction 

Deploy  attention 

Arouse  interest  and  motivation 

•  Focus  on  task 

•  How  new  skill  will  help  (may  be  implicit  from 
preview  or  may  need  context  of  larger  skill) 

Establish  instructional  purpose 

•  What  skill  to  be  learned  now  should  be  clear 

Preview  lesion 

•  Overview  what  will  be  learned  and  how 

Body 

Recall  prior  knowledge 

*  Point  out  the  known  skills  that  new  skill  uses 

Process  information 

*  Explanation  then  demonstration  or 

*  Explanation  and  demonstration  together 

*  Always  organize  by  steps  (subroutines)  of  the  skill 

Focus  attention 

•  Learner  activity  during  practice 

•  Special  attention  for  critical  skills  (dangerous,  etc.) 

nil           !          j      a  • 

Employ  learning  strategies 

*  Visualization  of  performance,  mnemonics,  analogies 

Practice 

*  Distribution  (whole/part)  Scheduling  (mass/spaced) 

•  Sufficient  for  automaticity  and  desired  skill  level 

Evaluate  feedback 

•  External:  suggestions,  comments 

•  Internal:  proprioceptive,  sensory 

conclusion 

Summarize  and  review 

•  Include  re-cap  of  the  steps  in  consolidating 
and  clarifying  fashion 

Transfer  knowledge 

•  Extended  practice  -  maintenance  of  proficiency 
fundamental  in  many  skills 

Remotivate  and  close 

•  How  apply  in  future;  when  use 

Assessment 

Assess  performance 

•  Observation  of  performance,  performance  rating 

Feedback  and  remediation 

*  Learner  needs  clear  idea  of  how  well  s/he  can 
perform  the  skill  and  what  to  do  next 

Psychomotor  learning:  Fitts  &  Posner,  1967;  Miller,  Galanter,  &  Pribram,  1960;  Oxendine, 
1984,  Robb,  1972;  Singer,  1982;  Instructional  considerations:  Practice:  Harrison  & 
Blakemore,  1989;  Rothstein,  Catelli,  Dodds,  &  Manahan,  1981;  Feedback:  Ho  &  Shea,  1978; 
Magil,  1985;  Newell,  1974;  Rogers,  1974;  Smoll,  1972 
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A  variety  of  conditions  and  attitudes  have  made  it  difficult  to  develop  and  use  evaluation 
schemes  for  "new  media."  Below  we  list  four  issues  which  pose  obstacles  to  effective 
evaluation. 

Continuous  change:  A  continuous  blizzard  of  technological  innovations 
often  blinds  designers  of  instructional  multimedia  to  the  fundamental 
questions  to  be  answered  by  formative  evaluation  in  the  course  of 
development.  As  a  result,  "evaluative  activities"  are  put  aside  in  order  to 
expend  effort  on  just  getting  the  multimedia  system  to  work  smoothly  in 
its  technological  environment,  and  on  insuring  that  the  system  is 
compatible  with  the  latest  software. 

Technology  focus:  The  delivery  of  multimedia  programs  can  require 
sophisticated,  non-standard,  andoften  costly  hardware  and  software. 
Thus  the  discussion  of  Multimedia  remains  focused  on  resolving 
hardware/software  issues:  who  can  tell  us  what  to  buy,  how  to  hook  it  up, 
and  how  to  keep  it  working.  The  necessary  concurrent  discussion  of 
learning  or  intellectual  activity  to  be  supported  by  multimedia  tools  pales 
by  comparison. 

Evaluation  in  isolation:  Evaluation  is  frequently  labeled  a  "phase" 
unto  itself,  residing  outside  of  the  central  development  activities  of 
analysis,  design,  and  implementation.  Frequently,  the  only  people 
involved  in  the  evaluation  are  the  prototypical  user  and  the  development 
teaai,  ignoring  the  interest  and  vested  interests  of  a  much  wider  group  of 
stakeholders,  further  isolating  the  process  by  keeping  it  out  of  a  real- 
world  environment.  Evlauation  is  conceived  of  as  a  measuring  of 
outcomes  and  products,  requiring  statistically  valid  instruments  and 
experimental  groups,  and  takes  place  long  after  key  decisions  have 
already  been  made  based  on  perceived  value  and  usability.  Thought  of  in 
this  way,  evaluation  seems  impractical. 

•    "Hyped  Media":  A  variety  of  claims  are  made  as  to  the  superiority  of 
instructional  multimedia  to  more  traditional  instructional  media  and 
methods.  Among  them  are  higher  grades,  improved  critical  thinking, 
accommodation  of  different  learning  and  cognitive  styles,  and 
improvement  of  teaching.  A  coherent  approach  to  confirming  these  claims 
has  yet  to  be  described  and  implemented. 

In  this  presentation  we  propose  a  framework  for  the  formative  evaluation  of  multimedia, 
describe  the  techniques  that  have  worked  best  in  our  software  development  efforts  at  Indiana 
University,  and  provide  examples  of  the  variety  of  ways  in  which  we  used  evaluation  results. 
Our  emphasis  is  on  what  is  workable  and  possible  to  do  given  the  traditonally  limited 
resources  of  the  instructional  designer  in  an  academic  setting.  Our  aim  is  get  the 
instructional  developer  to  use  a  variety  of  evaluation  techniques  more  frequently  and  with 
greater  confidence. 

Our  approach 

The  familiar  social  science  experimental  method  has  long  been  the  accepted  approach  to 
evaluating  the  effects  of  mediated  treatments  on  learning.  Scriven  (1987)  lists  these  steps. 
In  order  to  evaluate  a  program,  you 
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1.  Identify  the  goals  of  the  program 

2.  Convert  them  into  behavioral  objectives 

3.  Identify  tests  (or  construct  them)  that  will  measure  these  objectives 

4.  Run  these  tests  on  the  target  populations 

5.  v  runch  the  data 

6.  iteport  whether  or  not,  or  to  what  degree,  the  goals  have  been  met 

Characterizing  this  approach  as  the  "naive  social  science  model,"  Scriven  continues 

There  is  a  standard  set  of  about  fourteen  questions  that  need  to  be 
investigated  in  most  evaluations,  of  which  only  one  is  the  traditional 
investigation  of  alleged  or  hypothesized  effects  so  familiar  in  social  science 
research.  The  only  points  that  need  to  be  made  are  that  these  questions,  be 
they  concerned  with  cost  or  alternatives  or  ethics  or  unexpected  effects  or 
historical  background,  cannot  be  ignored;  and  that  a  systematic  approach  to 
them  is  possible,  with  about  the  same  chances  of  getting  an  answer  as  we  can 
expect  ih  the  usual  scientific  or  criminological  hunt  for  explanations  and 
theories,  (p.  65) 

In  contrast,  we  seek  to  arrive  at  the  answers  to  these  questions  from  within  the  development 
process  by  focusing  primarily  on  a  single  question:  to  what  degree  does  this  instructional 
multimedia  program  support  the  user's  activities/tasks  in  the  user's  task  environment?  For  if 
the  product  is  not  used  then  it  doesn't  matter  whether  it  creates  potential  for  or  enables  more 
or  better  learning.  Furthermore,  the  desired  learning  is  most  likely  to  take  place  far  from  the 
machine,  as  the  user  reflects,  applies,  and  synthesizes.  It  is  therefore  the  process,  or  the 
quality  of  the  mediation,  not  the  product  that  should  be  the  focus  of  instructional  multimedia 
evaluation  (Hutchings,  1992;  Marchionini,  1990). 

Our  views  on  an  appropriate  evaluation  model  for  multimedia  do  not,  however,  exist  in  a 
simple  bi-polar  opposition  to  what  Scriven  describes  as  the  "naive  social  science  model.  " 
Rather,  our  approach  can  be  seen  as  an  eclectic  one  moving  across  a  continuum  from  a 
management-oriented  model  (related  to  the  social  science  objectives-oriented  approach)  to 
the  naturalistic- participant  oriented  approach  at  the  opposite  end  (Worth en  and  Sanders, 
1987).  (See  Appendix  A).  According  to  Worthen  and  Sanders,  the  management-oriented 
approach  focuses  on  user  needs,  de-bugging,  and  evaluation  at  all  stages  of  development.  The 
naturalistic  approach  is  helpful  for  examining  innovations-in-use,  for  portraying  the 
complexities  of  and  educational  activity,  and  for  responding  to  an  audience's  needs.  From  our 
experience  of  both  success  and  failure  in  the  development  of  multimedia  programs  we  have 
seen  the  following  key  characteristics  emerge  in  our  approach  to  evaluation: 


A  focus  on  user  satisfaction 

What  do  we  mean  by  user  satisfaction  and  usability?  What  attracts  our  faculty  to  using 
multimedia  is  the  technological  reality  of  access  to  a  variety  of  media  via  the  computer.  The 
ability  to  draw  together  a  huge  volume  of  information  in  a  variety  of  formats,  makes  it 
possible  to  create  pnd  place  at  the  students'  and  teachers  disposal  elaborate 
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a  focus  on  user  satisfaction  (usability  and  valuing) 
integration  of  evaluation  into  the  design  process 
use  of  a  variety  of  techniques 
inclusion  of  a  range  of  stakeholders 
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learning/teaching  environments  not  possible  only  10  years  ago.  Thus,  the  issues  of  access  and 
control  are  paramount;  that  is,  the  program  must  facilitate  moving  about,  finding  things,  and 
control  appropriate  to  the  task  and  level  of  user.  Romiszowski  suggests  that  when  the 
purpose  of  the  program  is  to  provide  access  or  act  as  a  tool,  the  appropriate  evaluation 
approach  is  to  measure  user  satisfaction  (Romiszowski,  1990). 

To  the  definition  of  user  satisfaction  we  have  added  another  dimension:  valuing.  This  aspect 
is  based  on  the  simple  idea  that  a  tool  viewed  as  relevant,  critical,  and  of  wide  applicability  is 
a  tool  that  the  user  will  come  to  rely  on.  The  user's  'Valuing  statements"  comprise  a  face 
validity  of  the  evaluated  program.  Therefore,  in  all  of  our  evaluation  efforts  we  look  for 
valuing  statements. 

Integration  of  evaluation  into  the  design  process 

Evaluation  occurs  simultaneously  with  the  analysis,  design,  and  implementation.  Within  the 
large  scale  development  project  there  are  many  small  scale  design  processes  which  go 
through  a  full  problem  solving  process  for  purposes  of  exploring  options  and  developing  a 
fuller  understanding  of  the  requirements.  The  successive  phases  of  development-each  with 
concurrent  problem  analysis,  development  and  evaluation-become  increasingly  concrete 
(Goodrum,  Dorsey,  &  Schwen,  1994). 

Evaluation  could  be  said  to  lead  the  process.  The  creation  of  prototype  versions  of  proposed 
solutions  are  for  the  purpose  of  evaluation  and  retinement  of  specifications.  This  allows  users 
to  have  a  realistic  experience  for  basing  assessments  and  revisions.  Conceptual  prototypes 
allow  for  early  user  reaction,  feedback,  and  projection  of  consequences.  Working  prototypes 
allow  for  hands-on  use  in  the  context  of  the  task.  If  the  evaluation  process  begins  early  then 
parts  of  a  design  or  even  an  entire  design  can  be  discarded  before  time  investment  and 
escalating  commitment  prevent  such  corrective  action.  The  use  of  alternative  prototypes  in 
the  process  helps  keep  designers  and  users  from  locking  into  a  design  too  early.  Frequent 
user  evaluation  helps  insure  usability  and  provides  new  ideas  for  design,  content,  and 
evaluative  categories. 

Inclusion  of  a  range  of  stakeholders 

By  including  a  range  of  stakeholders  early  in  evaluative  activities  you  collect  ideas,  gain  buy- 
in  and  commitment,  and  avoid  unforeseen  technical  and  administrative  problems.  The  key 
stakeholders  most  often  missing  from  the  iterative  design  cycle  are  those  involved  in 
implementation  and  delivery,  for  example,  those  responsible  for  the  local  area  networks 
which  mUBt  have  the  capacity  and  flexibility  to  get  the  multimedia  program  to  the  user.  The 
developer  may  need  to  consciously  expand  his  or  her  idea  of  what  a  stakeholder  is.  To 
determine  stakeholders,  consider  who  or  what  could  block  the  user's  access  and  ablity  to 
work  with  the  program.  Another  way  to  find  all  the  stakeholders  is  to  think  of  everyone 
having  a  vested  interest,  and  let  the  non-stakeholders  self-select  out  of  the  group. 

Uhc  of  a  variety  of  techniques 

Kxamining  a  procogH  requires  gathering  snap-shots  at  various  stages  along  the  way,  calling 
for  u  concert  of  methods,  each  of  which  adds  "color"  to  the  description.  The  techniques  we 
have"  uwed  that  have  yielded  the  most  usable  data  have  their  roots  in  qualitative  inquiry. 
They  ore: 

•  Observation 
Self-report 
Interview 
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Peer  evaluation  (showing  and  telling  others) 


The  stage  of  development,  the  purpose  of  the  program,  the  target  user  and  other 
stakeholders,  help  determine  the  type  of  method  used  and  degree  of  formality  in  conducting 
the  evaluation  and  analyzing  the  results.  Another  way  to  approach  the  choice  is  to  ask 
(Knuscen,  Tanner,  and  Kirby,  1991): 

For  what  reasons  are  we  doing  an  evaluation? 
•    What  will  we  do  with  the  results? 
What  resources  do  we  have? 


Our  Experience 

The  nature  of  our  multimedia  development  has  been  a)  creation  of  presentation  packages  for 
faculty  clients  to  use  in  developing  multi-media  lectures  and  b)  creation  of  multimedia 
programs  for  students  to  use  in  networked  computer  clusters.  The  following  techniques  have 
proven  beneficial  to  our  evaluation  process,  especially  when  used  together  on  the  same 
multimedia  project. 

Client  questions.  Throughout  the  development  process  we  ask  the  client  to  compare  the  "old" 
way  with  the  "new  way"  of  doing  things.  What  was  wrong  with  the*  "old  way"  of  teaching  a 
topic?  Does  the  innovation  help?  How  does  it  help?  Could  you  use  this  in  other  courses?  Do 
you  think  that  you  could  take  on  more  of  the  development  yourself?  Client  questions  like 
these  keep  the  developer  and  client  in  a  critical  appraising  mode  focused  on  usability,  worth, 
and  value.  (See  Appendix  B.) 

Mock-ups.  Paper  mock-ups  may  be  used  at  a  variety  of  points  in  the  development  process. 
They  are,  in  a  sense,  like  a  structured  interview,  if  presented  as  a  question  rather  than  a  fait 
accompli.  At  the  beginning  of  the  process  they  are  an  inexpensive  and  simple  way  to  test 
ideas  for  fit  with  the  client's  or  user's  requirements.  They  can  provide  data  on  usability 
before  committing  to  a  design.  We  have  also  use  paper  mock-ups  in  mid-development  when  a 
project  has  foundered  or  been  dormant.  Removing  the  prbgram  from  the  "high  tech" 
environment  helps  refocus  everyone  on  task  and  usability,  and  the  underlying  reason  for 
creating  the  product.  It  may  also  invite  greater  participation  from  those  not  at  ease  with 
computers  and  from  those  who  may  be  new  to  the  project  if  it  is  being  picked  up  again  after  a 
period  of  time.  The  paper  mock  up  signals  an  openness  to  critical  evaluation.  (See  Appendix 
C.) 

Observed  initial  use.  This  may  be  done  as  formally  or  informally  as  resources,  data  needs  and 
stage  of  development  require.  After  4  months  of  development  on  one  program  we  conducted  a 
field  test  in  two  stages:  a  pre-field  test  review  conducted  by  instructional  developers,  two 
students,  and  a  faculty  member;  and  a  full  field  test  with  20  students  using  the  program  in  a 
network  cluster.  The  full  f  ield  test  involved  detailed  observation  of  two  students  and  a  survey 
of  the  whole  class.  The  tests  identified  current  and  potential  problems,  highlighted  the 
program's  positive  points,  provided  design  ideas,  and  involved  three  diferent  groups  of 
stakeholders.  It  di  d  require  more  coordination  than  some  of  the  other  methods,  however,  and 
preparation  of  the  evaluators.  (See  Appendix  D.) 

Field  trial.  To  be  effective,  the  field  trial  must  take  place  under  actual  conditions  that  test 
the  limits  of  the  products  caabilities.  Since  a  field  trial  is  conducted  during  actual  use,  care 
should  be  taken  to  provide  backup  and  support  in  case  the  system  fails.  Observations  may  be 
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highly  structured  or  subjective  evaluation  of  an  expert  (See  Appendix  E).  The  structure  and 
nature  of  the  observations  again  depend  on  what  data  is  needed  and  how  the  data  will  be 
used.  For  example,  are  you  still  looking  for  design  ideas,  are  you  firming  up  a  design  to  bring 
the  process  to  closure,  or  do  you  want  to  know  if  the  progam  is  usable  in  order  to  seek  more 
resources? 

Minute  paper.  Having  the  users/consumers  respond  in  writing  to  an  open-ended  question  can 
help  you  adjust  your  program  in  mid-course.  The  time  limit  forces  the  writer  to  divulge  only 
those  issues  of  primary  concern.  The  short  length  makes  it  possible  to  quickly  review 
responses  and  tabulate  results.  The  openness  allows  you  to  pinpoint  problems  you  may  not 
have  considered.  (See  Appendix  F.) 

Classroom  artifacts.  Student  and  faculty  client  questions,  criticisms,  compliments,  value 
statements,  problem  statements  are  all  evaluative  in  nature.  They  point  directly  and 
indirectly  to  places  where  the  program  either  performed  well  or  fell  short.  Electronic  mail 
now  helps  keep  a  record  of  them.  We  encourage  our  faculty  clients  to  keep  a  record  of  these 
artifacts  as  well  as  keep  a  record  of  mail  received  from  faculty.  Setting  up  a  "help"  phone 
service  during  specified  hours  can  also  help  capture  problems,  and  provide  evaluative  data. 

Focus  group.  The  focus  group,  conducted  by  a  person  perceived  as  impartial  and  open,  is  an 
efficient  way  to  interview  a  number  of  users  at  the  same  time.  We  have  had  success  with 
both  a  highly  structured  format  in  which  a  list  of  questions  was  prepared  in  advance,  and  an 
informal  format  in  which  small  groups  were  asked  one  or  two  open-ended  questions.  The 
more  informal  focus  groups  helped  in  defining  what  issues  would  become  important  later  in 
the  process.  The  structured  focus  group  was  conducted  at  a  later  stage  of  development,  at  a 
point  when  the  important  issues  had  been  more  clearly  identified  through  prior  evaluation. 
(See  Appendix  G). 

Survey.  A  survey  conducted  during  later  stages  of  development  may  complement  the 
structured  focus  group.  From  our  experience  we  have  identified  some  critical  survey 
categories: 

How  does  the  student  value  the  learning  experience? 

Was  the  experience  relevant  to  the  course  and  at  the  same  time  widely 

applicable? 

Was  the  program  useful  in  completing  the  assigned  task? 

How  does  the  student  compare  the  activity  with  activities  in  other 

courses? 

The  purpose  of  the  survey  is  to  gather  data  for  fine  tuning  performance  and  to  project  the 
"bottom  line"  results  of  using  the  program.  (See  Appendix  H). 

Peer  Evaluation.  Demonstrating  or  talking  to  peers  about  their  multimedia  programs  helps 
clients  remain  in  the  critical,  evaluative  mode  established  at  the  beginning  and  maintained 
during  the  design  process.  Peer  questions  are  opportunities  for  reflection,  generate  new 
ideas,  open  up  new  partnerships.  Peer  evaluation  may  take  place  among  other  stakeholders 
as  well,  with  the  same  effect  of  informing  current  and  future  design,  and  identifying  other 
criteria  by  which  the  program  might  be  judged.  (See  Appendix  I). 

Summary 

Our  approach  tc  the  evaluation  of  multimedia  contrasts  sharply  to  the  objectives-oriented 


tuteiul  science  evaluation  model  Rather  than  look  to  one  method  for  answers  to  all  of  our 
evaluation  questions  we  have  borrowed  heavily  from  naturalistic  and  participant-oriented 
cvalttftt  tott  approaches  to  obtain  the  answers.  Our  underlying  philosophy  toward  evaluation 
in  multimedia  development  process,  based  on  our  actual  practice  can  be  summed  up  this 
way: 

•  Kind  out  if  your  multimedia  helps  the  users  do  what  they  want  to  do. 

•  Kvaluating  usability  gets  you  most  (80%?)  of  what  you  need. 
Kvaluation  is  a  concurrent  process  through  all  stages  of  development. 
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Management  -oriented  Consumer-oriented        Naturalistic  and 

Participant-oriented 


Purpose; 

Providing  useful 
information  to  aid  in 
making  decisions 

Providing  information 
about  educational 
products  to  aid  decisions 
about  educational 
purchases  or  adoptions 

Understanding  and 
portraying  the 
complexities  of  an 
educational  activity, 
responding  to  an 
audience's  requirements 
for  information 

Character- 
istics 

Serving  rational 
decision-making, 
evaluating  at  all  stages 
of  program  development 

Using  criterion  checklists 
to  analyze  products, 
product  testing, 
informing  consumers 

Reflecting  multiple 
realities,  use  of  inds-  ive 
reasoning  and  d;  /, 
firsthand  experience  on 
site 

Criteria 

Utility,  feasibility, 
propriety,  ana  tecnmcai 
soundness 

Freedom  from  bias, 
lecnnicai  sou  nun  ess, 
defensible  criteria  used 
to  draw  conclusions  and 
make  recommendations; 
evidence  of  need  and 
effectiveness  are 
required 

Credibility,  Fit, 

auuitaoiuxy, 

confirmability 

Benefits 

Comprehensiveness, 
sensitivity  to 
informaiton  needs  of 
those  in  a  leadership 
position,  systematic 
approach  to  evaluation, 
use  of  evaluation 
throughout  the  process 
of  development,  well- 
operationalized  with 
detailed  guidelines  for 
implementation,  use  of 
a  wide  variety  of 
information 

EniphaniH  on  consumer 
information  needs, 
influence  on  product 
developers,  concern  with 
cost-iiffoctivonoHH  iiiul 
utility,  availability  of 
checklist** 

Focus  on  description  and 
judgment,  concern  with 
context,  openness  to 
evolve  evaluation  plan, 
pluralistic,  use  of  a  wide 
variety  of  information, 
emphasis  on 
unuerstanumg 

T.imit  atinnc 

Lu  III  11 I d  l»  IV  Ho 

p1  m  n  n  acic  nn 

organizational  efficiency 
and  production 

(""net  QTl/l  lafK  fix 
V/Uol  aiiu  Jam  \Jk 

sponsorship,  may 
suppress  creativity  or 
innovation,  not  open  to 
debate  or  cross- 
examination 

Mf4TirtiY*pr»tivp  tf^nnPTiPV  tn 

be  attracted  by  the 
bizarre  or  atypical, 
potential!  high  labor 
intensity  and  cost, 
hypothesis  generating, 
potential  for  failure  to 
reach  closure 

Adapted  from  Worthen,  B.  R.  and  Sanders,  J.  R.  (1987),  Educational  evaluation:Alternative 
approaches  and  practical  guidelines.  New  York:  Longman 

Client  Questions 


1.  What's  one  of  the  things  you'll  be  teaching  today? 

(looking  for  a  lecture  part  or  even  a  specific  point 
that  we  can  make  sure  we  capture  and  hilite) 

2.  How  did  you  teach  this  in  the  past? 

(what  were  the  frustration  or  limitation  of  teaching  it  that  way. 
i.e.,  what  was  wrong  with  that  way  of  teaching  it) 

3.  How  will  you  be  using  the  technology  to  help  you  teach  this  in  your  class? 

4.  How  difficult  was  this  to  create? 

(and  what  kind  of  help  did  you  need?) 

(do  you  see  yourself  doing  more  and  more  of  this  on  your  own?) 
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5.  Is  this  a  better  way  of  teaching?  Why?  (or  why  not) 


6.  What  other  courses  or  areas  in  your  field  would  this  be  useful  for? 
Insert  Graphic  entitled  "Mock-ups"  here. 


Insert  Graphic  entitled  "Observed  Initial  Use"  here. 

Field  Trial 

This  is  an  example  of  another  style  of  observation,  performed  at  a  later  point  in  development 
than  the  Observed  First  Use  example  in  this  packet. 

M  conducted  the  class  like  a  two-person  dance.  He  used  the  Komo  program  to  display  text 
mostly.  It  was  an  outline  he  traversed  to  give  students  an  idea  of  where  they  were.  It  seemed 
to  work  reasonably  well  because  he  didn't  go  too  deep  in  the  hierarchy.  He  has  someone  in 
the  back  controlling  two  slide  show  projectors.  Frequently  the  images  from  the  slides  and 
Komo  overlap.  Often,  the  person  controlling  the  slide  projectors  moves  the  images  to  keep 
them  from  falling  on  the  text  M  is  projecting.  Frequently,  the  tool  palette  from  the  program 
can  be  seen  over  the  slides.  M's  irreverence  is  plain-he  doesn't  seem  to  care  much  if  the 
slides  overlap  the  tool  palette  or  not.  The  person  in  the  back  swings  the  slides  back  and  forth 
trying  to  find  an  empty  space  for  them  as  M  turns  Komo  on  and  off.  There  is  no  "grid"~there 
is  no  place  where  text  always  comes  up  and  pictures  always  come.  The  presentation  is  like 
M-fluid,  ever  changing,  refusing  to  be  categorized  and  yet  paradoxically,  existing  within  the 
framework  of  a  highly  structured  outline.... 

...He  wants  to  be  able  to  access  things  quickly  in  any  way  he  wants.  The  interface  is 
extremely  important  to  him.  I  can  imagine  him  wanting  something  with  a  fluid  picture- 
showing  capability.  He  presses  "next  slide"  on  his  computer.  Up  pop  two  pictures.  He  points 
to  one  of  them  with  his  finger  on  a  stylus  and  drags  it  off  the  screen.  He  pops  up  some  text. 
He  presses  "clean  up"  and  all  of  a  sudden,  everything  fits/'  He  presses  another  button  and 
the  two  pictures  blow  up  to  fill  the  screen.  Another  button  and  the  pictures  disappear. 

Minute  Paper 

This  question  was  placed  on  an  overhead  projector.  Students  wrote  brief  answers  to  the 
question  vvxh  the  instructional  consulting  staff  then  reviewed  to  get  a  "reading"  on  how 
things  were  going  at  mid-semester.  The  purpose  was  to  spot  problems  and  fix  them  before  the 
end  of  the  semester. 


What  is  the  effect  of  the  technology  used  in  this  course  on  your 
learning? 

Insert  graphic  entitled  "Focus  Groups"  here. 
Insert  graphic  entitled  "Survey"  here. 

Peer  Evaluation 

A  faculty  developer  of  a  largo  multimedia  project  answer**  queHtiotiH  from  lu»r  peerH  iik  she 
domonRtratoK  her  program.  The  following  is  a  rough  Iraum  ript  from  the  video  recording  of 


H91 


her  demonstration. 


Faculty  developer:  An  advantage  of  this  format  ...it  really  does  give  student  access  to  a  much 
wider  array  data  than  ever  before  and  more  accessible  we've  started  tools  to  help  them  make 
those  connections  in  their  own  minds  in  different  sorts  of  ways  graphics,  text,  video  give  each 
student  a  little  bit  of  something 


...what  we're  trying  to  improve  upon  is  the  interactivity... 
Faculty  developer  stops  her  demonstration. 
Any  questions? 

Peer  question:  Do  students  type  their  answers  out? 
Delivery  issues  discussed 

Peer  question:  How  do  you  get  them  started  without  being  overwhelmed? 

Faculty  developer:  Simple  questions  and  self-help  tutorial.  Different  students  got  started  in 

different  ways. 

Disadvantage:  lots  of  data 
Advantage:  open  ended 

If  you  ask  the  right  questions. ..The  trick  is  the  interface  of  the  questions  you  ask. 
I've  challenged  graduate  students  with  this  data  set. 

I  wanted  them  to  use  this  as  a  tool  and  a  learning  resource  as  much  as  I  wanted  them  to  see 
use  it  as  a  text  so  I  didn't  allow  them  to  print  it  out. 

Peer  question:  You're  not  involved  with  authoring?  the  programming? 

Faculty  developer:  The  programmers  created  a  miniature  set  of  authoring  tools 

that  have  ullowed  me  to  -  created  a  template  -- 1  create  the  images  and  can  very  quickly... 

I  put  it  together  mostly,  I  haven't  done  the  scripting.  (Faculty  developer  goes  into  stack  and 
adds  to  it.  She  opens  a  palette  to  choose  a  card  format,  then  she  brings  in  a  picture,  types  in 
thetoxt) 

That's  how  I  put  it  together.  It's  been  very  easy  to  put  it  together.  I  know  a  lot  more  about 

hyporcurd  than  I  ever  did. 

n  cut  and  paste  and  a  content  problem  for  me 

Poor  comment:  Very  flexible  architecture  overall. 

Faculty  developer:  Next:  use  this  as  a  presentation  tool 
Astound  -  allow  you  to  select  a  few  elements  -  I'll  be  working 
on  that  next  semester 

Peer  question  on  copyright  issues. 

Faculty  developer:  Copyright  issues  -  got  to  mail  out  the  letters!  I've  tried  to  replace 
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copyrighted  material  with  my  own  material 
any  color  stuff  is  mine. 

I  like  to  doodle,  so  I  can  always  create  something  myself. 
Prototype  on  a  CD 

At  each  stage  we've  gotten  student  evaluate  information 
I've  been  astonished  how  resilient  the  students  are. 
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The  study  of  motivation  has  long  been  a  neglected  area  in  instructional  technology. 
The  emphasis  on  promoting  effectiveness  and  efficiency  in  instructional  design  often  excludes 
concerns  about  the  appeal  of  instruction.  Traditionally  handicapped  by  the  lack  of  theory 
and  lack  of  measurements  dealing  with  motivation,  instructional  designers  have  assumed 
that  good  quality  instruction  will  in  itself  be  motivating  (Keller,  1983). 

However,  research  in  the  field  of  intrinsic  motivation  over  the  last  decade  has  yielded 
insights  which  can  be  applied  to  problems  in  instructional  design.  This  research  has 
resulted  in  the  development  of  Flow  Theory,  a  model  which  explains  the  structure  of  optimal 
experiences  and  the  dynamics  of  intrinsic  motivation. 

This  paper  will  provide  a  synthesis  of  Csikszerrtmihalyi's  Flow  Theory  of  intrinsic 
motivation  with  Keller's  Motivational  Design  Theory  of  Instruction.  The  conceptual 
integration  of  these  two  theories  will  be  the  basis  for  presenting  prescriptions  aimed  at 
increasing  the  level  of  intrinsic  motivation  in  instruction. 

Theoretical  framework 

The  flow  phenomenon  is  best  described  in  terms  of  its  two  parameters,  challenge 
and  skills.  When  these  two  factors  are  in  balance,  people  feel  that  they  have  the  skills 
necessary  to  achieve  a  challenging,  but  realistic,  goal  (Csikszentmihalyi,  1982).  A  challenge 
is  perceived  as  an  activity  that  has  clear  goals  and  requires  an  investment  of  effort  in  order 
for  the  goal  to  be  achieved.  Skills  are  those  efforts  that  an  individual  needs  to  apply 
towards  the  achievement  of  the  goal.  Implicit  in  this  model  is  the  characteristic  of  feedback, 
which  allows  an  individual  to  track  his  or  her  efforts  in  achieving  the  goal. 

Csikszentmihalyi  (1993)  states  that  flow-like  activities  have  four  main 
characteristics: 

(1)  they  have  concrete  goals  and  manageable  rules,  (2)  they  make  it  possible  to 
adjust  opportunities  for  action  to  our  capabilities,  (3)  they  provide  clear  information 
about  how  well  we  are  doing,  and  (4)  they  screen  out  distractions  and  make 
concentration  possible,  (p.  xiv) 

Flow  experiences  are  informative  in  nature,  that  is,  they  are  clearly  hounded  by  rules 
and  goals,  yet  are  flexible  enough  to  allow  an  individual  to  exert  effort  and  experience  the 
results  of  those  efforts. 

The  emotional  effects  of  attempting  to  achieve  any  goal  can  be  classified  according  to 
the  ratio  of  challenge  to  skill  an  individual  applies  to  the  task.  These  ratios  are  termed 
vhannels"  and  are  grouped  into  classes  and  defined  by  Csikszentmihalyi  St 
'  -'Uszentmihnlyi  (1988),  as:  flow,  in  which  the  level  of  challenge  is  equal  or  slightly  above 
ine  level  of  skill  required  to  achieve  the  task;  anxiety,  in  which  the  level  of  challenge  is 
greater  than  the  amount  of  skill  necessary  for  the  task;  apathy,  in  which  both  the  level  of 
challenge  and  the  level  of  skill  is  low;  and  boredom,  in  which  the  level  of  challenge  is  lower 
than  the  amount  of  skill  an  individual  can  bring  to  the  task  (see  figure  1). 

Flow  is  perceived  as  a  positive  experience  because  the  mastery  of  an  achievable 
challenge  stretches  an  individual's  capabilities  by  promoting  the  development  of  new  skills 
and  increasing  self-esteem  and  personal  complexity  (Csikszentmihalyi  &  LcFovre,  1989).  In 
addition,  remaining  in  the  state  of  flow  requires  further  increases  in  the  complexity  of  the? 
challenge,  thereby  promoting  growth. 

While  in  a  state  of  flow,  people  often  report  feeling  more  active,  alert,  concentrate  i, 
happy,  satisfied,  and  creative  (Csikszentmihalyi  &  LeFevre,  1989).  Flow  produces  a  st-ise 
of  focused  concentration  described  as  the  "merging  of  activity  and  awareness"  which 
Csikszentmihalyi  claims  "produces  harmony  within  the  self  (Csikszentmihalyi  & 
Csikszentmihalyi,  1988). 
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Figure  1:  Tin*  "fuiir-rliiinnt1!"  flow  model.  (Adapted  from  Massimini  and  Carli,  1986,  as 
ritftl  in  ('HiliH/.rnlmihalyi  &  ('Hiknzentmihalyi,  1988.) 

In  tonus  of  instructional  design,  flow  theory  emphasizes  the  importance  of  three 
iIi'huim  varinlili'8,  i\\m\nt  means  and  feedback.  Instructional  designers  promote  the  use  of 
mntriiHiiiiwil  nh.HM'1  iv«»H  iik  tin*  backbone  of  the  development  process.  The  emphasis  in  flow 
I  hi'iM  V  mi  Hit*  nii|Mirt fiiu-c*  of  Hear  gunta  reinforces  this  basic  prescription.  In  addition,  the 
importance  In  How  tliooi  v     communicating  clear  means  to  achieve  goals  is  the  ir.<un 
|tiir|iiiHi*  cil  tlii1  iiifiii v  instruct  inmil  design  theories  and  models  that  have  developed  in  our 
I  mid   Anil  (hr  (•iii|iImh1h  mi  rlnir  feedback  in  flow  theory  has  also  been  an  issue  in 
matt  net  imuil  ilcHign  since  tin*  days  of  Skinner  and  the  development  of  programmed 
instruction 

TIicmi'  three  environmental  characteristics  of  flow  provide  a  theoretical  congruence 
hot  wi'i»M  f  *Hikt4/i*nt  iniliiilvi'H  theory  and  the  practice  of  instructional  design.  Flow  theory  ran 
provide  liiHt  I'Mrt  luniil  designers  with  n  clearer  understanding  of  the  motivational  aspectH  of 
hist  rncl  Inn  while  iiIho  providing  iik  with  a  set  of  criteria  for  evaluating  the  appeal  of 
Distraction 

Tin*  first,  and  perhaps  only  II)  theory  to  explicitly  address  the  use  of  motivation  is 
John  Ki'IIi'i'h  hint  ivatiuiiiil  design  theory  (Reigeluth,  1987).  This  theory  is  integrative  in 
nature  anil  Ih  Intended  In  In*  used  in  conjunction  with  other  instructional  design  theories. 
Tin*  llieorv  linn  I ii  it  I  i  a  descriptive  component,  the  "Motivational  Design"  theory,  anil  a 
prescriptive  nrnpoiient,  known  iih  the  ARCS  (Attention,  Relevance,  Confidence,  and 
Hat  tnfiirt  inn  I  model  of  IiiHtrurtinnal  strategies  (Keller  &  Kopp,  1987). 

In  amilvzliiM  iiikI  Hynthi'HiziiiK  the  research  on  motivation  in  education  Keller  lias 
identified  throe  major  catoKnrlcH  which  form  the  backbone  of  his  theory:  effort,  performance, 
Mint  miiniMpii'iiroH    In  addition,  each  of  those  categories  has  both  personal  and 
environmental  factor*  which  affect  them, 

This  synthesis  Ikih  n  I  in  a  ic  assumption;  that  the  intrinsic  motivational  theory  of 
flow  hIioiiIiI  In*  thought  iif  iih  a  subset  of  Keller's  larger  theory  of  motivation.  The  goal  was  to 
incorporate  flow  theory  within  t  he  throe-categories  framework  of  Keller's  theory. 

Km  in  thin  perspective  then,  Keller's  category  of  effort  corresponds  to  the  challenge 
parameter  of  tin1  flow  model.  Thin  in  based  upon  the  assumption  that  an  individual's 
amount  nl  effort  is  at  least  partially  a  function  of  their  perception  of  the  challenge. 

Tin1  performance  category  of  Keller's  theory  corresponds  to  the  skills  parameter  of 
flow,  if  wo  assume  t  hat  an  individual's  performance  is  dependent  upon  an  adequate  level  of 


<>97 


skills  to  meet  the  challenge. 

Finally,  thr  const^ui'iin's  I'uti'gory  of  Keller's  theory  would  correspond  to  the  function 
of  feedback  in  the  flow  model.  l'Vi'dhni'k  ih  used  in  the*  Flow  model  to  adjust  the  balance  of 
challenge  and  skills  to  vvvtitv  optimal  experiences.  Interpreted  from  Keller's  framework,  the 
consequences  of  an  individual's  performance  functions  as  a  feedback  mechanism,  therefore 
allowing  the  individual  to  evaluate  performance  Mid  modify  behavior  if  necessary  (see  figure 

In  addition,  personal  anil  environmental  factors  have  been  derived  from  Plow  Theory 
which  correspond  to  these  elements  in  Keller's  theory. 


Motivational  design:  bated  on  Killer 


Intrinsic  motiavtional  design:  based  on  CsikaszentnritiBlyi 

Figure  2:  The  conceptual  fit  between  Keller's  and  Csikszentmihalyi's  theories,  showing 
intrinsic  motivation  as  a  subset  of  motivation 


This  synthesis  provides  a  number  of  advantages  to  the  instructional  designer,  First, 
it  provides  a  unified  conceptual  framework  for  the  motivational  design  of  instruction,  which 
not  only  takes  advantage  of  the  latest  research  in  intrinsic  motivation  but  also  fits  within 
our  own  conceptual  frame  of  reference.  Second,  it  can  help  clarify  the  issues  in  intrinsic 
motivation  by  defining  the  chief  parameters  and  relationships  within  the  theory.  Third,  ns 
with  Keller's  theory,  it  leads  to  predictions  about  the  relationships  between  intrinsic 
motivation,  learning,  and  performance.  And  finally,  it  can  be  used  as  a  basis  for  developing 
prescriptions  for  the  design  of  intrinsic  motivation  in  instruction. 

Methodology 

As  an  initial  attempt  to  derive  these  prescriptions,  an  extensive  literature  review 
was  conducted  on  the  topic  of  intrinsic  motivation.  This  literature  review  covered  a  wide 
variety  of  authors,  operating  from  a  number  of  theoretical  viewpoints.  Their 
recommendations  were  interpreted  from  the  viewpoint  of  flow  theory,  and  specific 
prescriptions  were  derived  which  fit  into  the  framework  of  the  theory. 

The  prescriptions  are  gri  aped  into  three  major  categories  which  correspond  to  the 
three  variables  presented  in  the  synthesis:  challenges  (incorporating  goals),  means  (including 
skills),  and  feedback. 
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Challenge  Strategies 

The  first  set  of  strategies  listed  under  CI:  "Challenges/Skills"  (see  table  1)  are  based 
upon  Csikszentmihalyi's  (1990)  identification  of  tin?  firnt  major  element  of  flow.  It  is 
important  for  the  designer/teacher/manager  to  clearly  identify  the  goals,  means,  and 
boundaries  of  the  activity.  These  provide  the  "rules  of  the  game"  that  students  will  be 
operating  under.  It  is  these  very  constraintB  that  can  he  liberating  for  a  learner  because 
they  reduce  ambiguity  and  doubt.  Once  the  challenge  has  been  get  however,  learners  should 
be  given  the  maximum  amount  of  freedom  to  explore  within  the  boundaries  of  the  activity. 
The  idea  is  to  clearly  define  goals  and  final  outcomes,  but  not  to  micromanage  the  process 
for  each  individual. 

The  strategies  under  C2  deal  with  the  need  to  design  challenges  that  are  optimal  in 
nature,  that  is,  activities  that  are  difficult,  but  achievable  by  the  target  group  (Reeve,  1992). 
Since  individual  differences  may  vary  considerably,  challenging  activities  should  be  designed 
with  a  variety  of  difficulty  levels  so  that  individual  learners  can  enter  them  at  an 
appropriate  level.  Furthermore,  the  activity  must  contain  an  element  of  accelerating 
difficulty  to  compensate  for  the  growth  in  the  learner's  skills  (Csikszentmihalyi,1990). 
Feedback  that  is  directly  linked  to  performance  is  another  useful  element  in  designing 
optimal  challenges  because  it  allows  the  learners  to  systematically  judge  their  progress  and 
fosters  an  increased  sense  of  competence  (Deci  &  Ryan,  1985). 


Table  1:  Challenge  Strategies 


CI.  Challenge/Skills 

C 1 . 1     Create  challenging  activities  that  require  the  application  of  of  skills. 
Cl  .2     Clearly  communicate  the  goals  of  the  activity,  and  the  rules  bounding 
the  activity. 

Cl.3     Clearly  communicate  the  means  to  reach  the  goal  by  identifying  pre- 
requisite skills, 

C2,  Optimal  Challenges 

C2.1  Create  challenging  activities  that  are  optimal  in  nature. 

C2.2  Create  activities  with  varying  degrees  of  difficulty. 

C2.3  Create  activities  with  accelerating  difficulty  levels 

C2.4  Utilize  clear  and  informative  feedback  to  communicate  performance. 

C3.  Goal  Setting 

C3.1     Create  goals  for  the  activity  which  are  interesting,  challenging,  and 

clearly 

communicated. 
C3.2     Create  activities  that  allow  for  goal  setting. 
C3.3     Utilize  short-term  goals  for  individuals  whose  initial  intrinsic 

motivation 

is  low. 

C3.4     Utilize  long-term  goals  for  individuals  whose  initial  intrinsic 
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motivation  in 


(Jon  I  setting  in  the  concern  of1  the  strategies  listed  under  C3.  Within  the  boundaries 
of  tin1  activity,  personal  goal  Hotting  should  be  encouraged.  However,  guidance  as  to  the  type 
of  goals  should  Ih*  provided  based  upon  the  learner's  initial  level  of  intrinsic  motivation 
(Vallcraml  ft  nl.,  11)87).  Learner*  with  high  levels  of  intrinsic  motivation  should  be 
encouraged  to  set  their  own  goals,  particularly  long  term  goals;  learners  with  low  levels  of 
in  riiinir  motivation  may  require  a  series  of  short  term  goals  and/or  extrinsic  rewards. 

Means  Strategies 

Tin1  second  category  of  strategies  deals  with  Ihe  means  of  achieving  the  goals  set  up 
by  the  first  set  of  prescriptions.  This  category  has  been  broadened  from  CsiksKentmihalyi's 
emphasis  on  individual  skill  development  to  also  include  environmental  aspects  which  help 
create  a  supportive,  informational  context  (see  table  2). 

According  to  cognitive  evaluation  theory  (Deci  &  Ryan,  1985;  Vallerand,  et  nl.,  1987) 
an  informational  context  is  necessary  to  promote  intrinsic  motivation.  The  strategies  under 
Ml  deal  with  approaches  toward  developing  an  informative  learning  environment.  These 
strategies  are  used  to  encourage  the  development  of  individual  initiative,  because  intrinsic 
motivation  depends  upon  a  perception  personal  control  and  a  sense  of  achievement  (Reeve, 
1992;  Burger,  1985). 

The  M2  strategies  address  environmental  parameters  that  encourage  the 
development  of  learners'  skill  levels.  These  are  based  upon  Csikszentmihalyi's  seventh 
element  of  flow  (1990)  along  with  Gagne's  (1965)  emphasis  on  the  identification  of 
prerequisite  skills  and  knowledge  as  the  basis  for  instructional  design.  Strategy  M2.3 
relates  to  those  metacognitive  skills  derived  from  the  principles  of  flow  theory 
(Csikszentmihalyi ,  1 990) . 


Table  2:  Means  Strategies 


Ml.  Informational  Context 

Ml.l    Create  an  informational,  rather  than  a  controlling,  context  for  the 
activity. 

Ml. 2    Within  the  parameters  of  the  activity,  encourage  nclf-dctcrmination 
by  maximizing  personal  choice  on  the  selection  of  goals. 
M        Support  feelings  of  autonomy  and  freedom,  and  encourage  a  tank- 
orientation  by  allowing  learners  to  direct  their  own  practice  and  to  meet  their 
own  stiihdanls  of  excellence. 

M2.  Skill* 

M'2.1    Construct,  activities  that  allow  for  the  development  of  skills  that 
reduce  the  margin  of  error, 

M2.2    Teach  and  practice  all  necessary  pre-requisitc  skills 
M2.il    Kncourage  learnerK  to  monitor  their  own  feelingH  of  anxiety,  apathy 
and  boredom,  and  to  use  these  feelings  as  feedback  for  adjusting  the 
challenge  level  of  the  activity. 
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Feedback  Strategics 

The  strategies  dealing  with  feedback  under  section  Pi  tiro  intended  to  hi1  general 
heuristics  dealing  with  the  overall  feedback  functio.  (see  table  3).  Strategics  Kl  .3  through 
F1.5  are  based  upon  recommendations  made  by  Vallerand  et  al.  (1 987).  The 
recommendation  on  the  use  of  extrinsic  rewards  for  low  initial  intriniuc  motivation  is  from 
Reeve  (1992), 

Positive  verbal  feedback,  if  it  is  used  in  a  context  of  self-determination,  can  enhance 
perceived  competence,  while  negative  feedback  often  decreases  competence  if  it  is  perceived 
in  a  controlling  manner.  These  recommendations  are  derived  from  Deci  &  Hyan  (1985). 

The  concept  of  naturally  occurring  feedback,  and  the  two  specific  forms  of  it 
recommended  in  strategies  F3.2  and  F3.3,  are  again  based  upon  Deci  &  Ryan  (1985). 

The  prescriptions  on  both  direct  and  indirect  competition  are  also  from  Deci  &  Ryan, 
(1985).  It  should  be  noted  that  all  forms  of  competition  should  hu  used  carefully. 
Direct  competition  should  be  used  only  for  extrinsic  purposes,  if  at  all,  as  it  tends  to  have  a 
de-motivating  effect  on  those  who  persistently  lose. 

Application 

Some  instructional  strategies  which  seem  to  naturally  incorporate  some  of  these 
prescriptions  include  hypermedia  applications,  simulations  and  games,  and  adaptive 
instructional  systems. 

A  high  level  of  self-determination  and  personal  choice  is  a  key  feature  of 
hypermedia  systems,  and  in  terms  of  intrinsic  motivation,  it  is  their  strongest  characteristic. 
Of  the  various  types  of  applications  being  reviewed,  hypermedia  provides  the  greatest 
amount  of  choice,  however,  these  systems  also  require  the  users  to  set  their  own  goals  and 
define  their  own  feedback.  The  only  goal  inherent  in  these  systems  is  the  search  and 
retrieval  of  information.  Their  use  in  a  broader  educational  context  requires  that  goals  be 
defined  either  by  the  teacher  or  by  the  learner.  The  same  problem  can  be  seen  in  relation  to 
the  feedback  function,  which  consists  largely  of  answering  the  question  "Did  I  find  the 
information  I  was  seeking?".  However,  the  exploratory  nature  of  most  hypermedia  systems 
allow  learners  to  seek  out  increasingly  difficult  levels  of  material.  When  used  appropriately, 
these  systems  can  provide  a  challenge  which  is  optimal  in  nature. 


Table  3:  Feedback  Strategies 


PI.  General 


Pi.  1  Feedback  should  be  concrete,  immediate,  and  logically  related  to 
the  goal. 

F 1 .2     Utilize  feedback  for  informational,  rather  than  controlling 
purposes. 

Pi. 3    Offer  feedback  and  advice,  but  don't  impose  it. 
P 1 . 4    Create  appropriate  pacing  of  feedback  for  each  individual. 
Fl.5     Provide  a  supportive  context  by  de-emphasizing  pressure  to 
perform  or  compete. 

Fl  .6     Utilize  extrinsic  rewards  only  if  there  is  low  initial  intrinsic 
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motivation. 


V2  Vurbul  Ftmlback 


F2. 1  Utilize  creditable  and  informative  verbal  feedback  to  promote  the 
perception  of  competence. 

F2.2    Avoid  negative  verbal  feedback  which  can  be  perceived  as  a 
judgement  of  incompetence. 


F3. 1     Design  activities  that  incorporate  feedback  which  is  naturally 

obtained  during  the  process  of  achieving  the  goal. 

F3.2     Utilize  self-evaluation  feedback  whenever  possible. 

F3.3    When  concentrating  on  pre-requisite  skills,  compare  performance 

against  a  standard. 


F4.  Competition 

F4. 1     Utilize  indirect  competition  if  it  can  be  perceived  as  being 
informational. 

F4.2    Avoid  the  use  of  direct  competition  if  it  is  perceived  as  controlling. 


Games  and  simulation  incorporate  to  a  great  degree  the  principles  of  intrinsic 
motivation.  By  their  very  nature,  games  and  simulations  require  a  clearly  defined  goal,  and 
the  rules  for  play  or  operation  must  be  clearly  defined.  The  challenge  inherent  in  a  game  or 
simulation  can  be  made  optimal  by  merely  changing  the  variables  or  the  rules  to  make  the 
activity  more  difficult  as  play  progresses. 

Skill  development  within  the  simulation  game  is  provide  by  practice  and  strategy 
development.  There  is  often  a  high  degree  of  self-determination  and  personal  choice  within 
the  boundaries  of  the  activity,  with  a  strong  orientation  toward  the  task,  that  is,  the  desire 
to  win  the  game. 

Feedback  from  a  simulation  game  is  almost  always  concrete  and  immediate,  and  is 
always  naturally  occurring  within  the  context  of  the  activity.  There  is  also  a  strong  element 
of  competition,  which  can  be  both  direct  and  indirect  in  nature.  This  helps  create  an 
atmosphere  that  is  informative  in  nature,  in  which  performance  is  evaluated  directly  in 
terms  of  the  task. 

However,  for  educational  purposes,  games  and  simulations  must  be  integrated  into 
a  course  with  supporting  instruction  both  prior  to  and  after  thn  activity.  Standing  alone, 
they  may  provide  entertainment,  but  they  do  not  ensure  the  the  acquisition  of  specific 
knowledge  or  skill. 

Adaptive  Instructional  Systems  hold  out  much  promise  in  their  ability  to  incorporate 
the  principles  of  intrinsic  motivation  into  instruction.  As  can  he  scon  from  the  research 
reviewed,  intrinsic  motivation  depends  upon  a  fine  balance  between  clearly  defining  and 
structuring  the  learning  activity  and  allowing  the  teamer  a  high  degree  of  self-determination 
and  personal  choice. 

The  purpose  of  an  adaptive  system  is  to  adapt  the  style,  sequence  or  content  of  the 
instruction  to  the  learners1  individual  differences.  These  differences  include  aptitude,  prior 
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knowledge,  cognitive  styles,  and  personality  characteristics,  such  as  motivation,  levels  of 
anxiety,  locus  of  control  and  risk  taking.  Research  has  shown  that  aptitude,  particularly 
intelligence,  and  prior  knowledge  are  the  two  strongest  predictors  of  performance  in 
instruction.  Among  personality  characteristics,  motivation  is  the  difference  that  most  affects 
learning  (Jonassen,  1988;  Carrier  &  Jonassen,  1988). 

An  adaptive  system  that  could  accurately  adjust  the  difficulty  level  of  the  instruction 
for  a  particular  learner  as  they  were  using  it  would  be  able  to  maintain  an  optimal  level  of 
challenge.  The  degree  of  flow  experienced  by  a  learner  could  be  maximized  in  a  system 
which  queried  the  emotional  state  of  a  learner  during  use  (i.e.  whether  a  user  felt  bored, 
anxious,  etc.),  or  which  analyzed  and  used  content-centered  responses  to  gain  information 
about  the  user's  motivational  state.  This  information  could  then  be  used  by  the  system  as 
the  basis  for  further  adapting  the  instruction. 

Conclusion 

It  should  also  be  noted  that  these  prescriptions  are  preliminary  and  require  further 
research  and  development.  A  process  of  expert  review,  brainstorming,  and  refinement  of 
strategies,  as  recommended  by  Keller  (1987a),  should  be  employed  to  further  extend  this 
model.  In  addition,  the  author  is  currently  conducting  a  research  study  to  gather  empirical 
evidence  in  support  of  these  prescriptions. 

The  significance  of  this  approach  is  that  it  attempts  to  begin  an  investigation  into  an 
aspect  of  education  and  training  which  is  critical  but  often  neglected.  Motivation,  it  can  be 
argued,  is  one  of  the  most  important  factors  in  education.  A  highly  motivated  student  will 
strive  to  overcome  many  limitations  in  order  to  achieve  his  or  her  goals.  An  unmotivated 
student,  however,  will  achieve  little  in  even  the  most  resource  rich  environment.  Initial 
motivation  of  an  individual  towards  a  topic  depends  upon  many  personal  and  psychological 
factors.  However,  continued  engagement  in  the  topic  can  depend,  to  a  large  extent,  on  how 
that  activity  is  structured.  Flow  theory  identifies  those  structural  variables  which  can  be 
manipulated  by  an  instructional  designer  to  increase  the  likelihood  that  a  learner  will  be 
motivated  on  an  intrinsic  level. 
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Instructional  design  was  born  in  the  twentieth  century,  emerging  from  the  theoretical 
roots  of  other  disciplines.  The  foundations  of  macro-design  processes  are  in  general  systemH 
theory  and  are  apparent  in  the  instructional  systems  design  (ISD)  models,  as  exemplified  by 
the  Dick  and  Carey  (1990)  model.  The  micro-design  strategies  are  based  primarily  upon 
learning  and  instructional  theory  as  exemplified  by  Gagne's  Events  of  Instruction.  Designers 
use  both  macro-  and  micro-models*  to  provide  direction  for  their  work  and,  as  Duffy  and 
Jonassen  (1991)  have  noted,  design  endeavors  are  always  theory-dependent  even  though  the 
designers  may  not  be  explicitly  aware  of  the  application.  The  task  of  this  paper  is  not  to 
describe  theory  changes  in  the  field,  but  rather  to  anticipate  the  nature  of  the  influence 
theory  will  have  upon  the  design  models  of  the  twenty-first  century,  based  upon  the 
emerging  theoretical  developments  of  today. 

Efforts  will  be  made  to  distinguish  between  model  enhancements  which  are 
generated  primarily  from  new  theory  and  research,  as  opposed  to  model  enhancements 
which  are  the  product  of  changing  practitioner  traditions,  such  as  those  prompted  by 
emerging  constraints  in  the  workplace,  or  the  impact  of  the  new  technologies.  Admittedly, 
this  demarcation  is  not  always  clear  since  practice  and  theory  are  often  interrelated,  and 
ISD  models  themselves  are  typically  a  result  of  the  combination  of  abstract  theoretical 
principles  and  analyses  of  practitioner  experience.  At  times,  this  blurring  of  theoretical  and 
practical  innovations  is  evidenced  by  emerging  models  which  seems  to  represent  the  impact 
of  new  thinking,  rather  than  actual  theory. 

The  anticipated  model  changes  discussed  in  this  paper  are  the  result  of  the 
disparate  theoretical  thinking  in  areas  such  as  chaos  theory,  constructivism,  situated 
learning,  cognitive  learning  theory  and  general  systems  theory.  In  addition,  examples  of  new 
ideas  which  bridge  theory  and  practice  will  also  be  addressed.  Such  topics  include  designer 
decision  making  research,  performance  technology,  and  the  quality  movement. 


In  spite  of  tho  criticism  in  some  quarters  of  traditional  ISD  models  (see  Richey,  1993 
for  a  review  of  mich  criticism),  it  is  difficult  to  imagine  that  they  will  not  continue  to  be  a 
driving  farce  among  practitioners  in  the  next  decade.  It  is  likely,  however,  that  they  will 
change.  These  trends  appear  to  be  directed  generally  toward  altering  the  traditional 
linearity  of  the  design  process,  and  specifically  toward  the  expansion  of  front-end  analyses, 
and  the  formative  and  summative  evaluation  procedures.  The  fact  that  these  aspects  of  the 
design  process  are  topics  of  discussion  is  not  unusual;  the  nature  of  the  discussion  is 
unusual,  because  of  the  current  efforts  to  reduce  design  cycle  time  to  accommodate  time  and 
money  constraints  (Dick,  1993a;  Gustafson,  1993).  Needs  assessment  and  evaluation 
procedures  are  typically  the  first  design  elements  to  be  sacrificed  by  practitioners,  and  the 
linear  structure  facilitates  the  use  of  rapid  prototyping  and  other  techniques  used  to  reduce 
cycle  time.  Nonetheless,  these  are  the  areas  impacted  most  by  current  theoretical  trends. 

Nonlinear  Design  Models  of  the  Future.  While  most  current  ISD  models  are  not 
explicitly  constructed  for  use  only  in  a  linear  fashion,  it  is  nonetheless  the  typical  pattern  of 
use.  Criticism  of  this  practice  is  becoming  more  widespread,  and  it  is  sustained  by  the 


'In  some  parts  of  the  literature  the  terms  "macro-design"  and  "micro-design"  refer  to  the  size  of  the 
instructional  unit,  with  macro  referring  to  program  or  curriculum  design  and  micro  referring  to  lessons  or 
modules.  This  is  not  the  orientation  here;  rather  I  am  distinguishing  between  overall  design  and 
development  processes,  and  strategy  selection  and  sequencing. 
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growing  interest  in  chaos  theory,  cognitive  advancements,  and  a  new  look  at  general  systems 
theory.  Quite  briefly,  chaos  theory  posits  the  existence  of  order  within  apparently  chaotic  or 
seemingly  random  systems  (You,  1993).  Chaos  theory,  especially  when  examined  in  detail, 
can  have  great  implications  for  instructional  design.  You  (1993)  discusses  the  notion  of 
linearity  within  the  ISD  process  and  the  support  chaos  theory  gives  to  changing  from  linear 
to  nonlinear  design  models.  He  asserts  that  nonlinear  ISD  models: 

assume  a  more  holistic  orientation,  rather  than  one  of  mutual  causality; 
are  able  to  better  accommodate  those  factors  in  a  given  situation  which 
can  interfere  with  the  prescribed  design  processes;  and 
reflect  the  dynamic  and  unpredictable  aspects  of  the  learning  process. 
Due  to  this  nonlinear  process,  the  while  (i.e.  the  final  instructional  product)  becomes  more 
than  the  sum  of  its  parts.  This  position  stems  from  a  view  of  current  linear  models  as 
reductionist  in  nature,  implying  a  cause  and  effect  relationship  between  the  various  stages 
in  the  ISD  process.  In  other  words,  the  output  of  one  phase  becomes  the  input  of  the  next. 

Support  for  nonlinear  design  models  also  is  appearing  from  traditional  sources  of 
design  theory  --  from  cognitive  psychology  and  general  systems  theory.  Tennyson  (in  press) 
sees  emerging  design  models  (ISD^,  or  fourth  generation  instructional  design)  as  a  reflection 
of  the  cognitive  science  movement  Specifically,  this  new  form  of  ISD  is  "a  dynamic,  problem 
solving  approach... controlled  by  contextual  or  situational  problem  solving  evaluations"  (p. 
10).  This  new  ISD  is  a  flexible,  dynamic  process  with  its  various  component  activities 
"connected  by  context  rather  than  by  predefined  ID  processes"  (p.  11).  The  activity  domains 
(analysis,  design,  production,  implementation,  and  maintenance)  are  based  upon  a 
situational  evaluation.  The  domains  are  viewed  as  overlapping  activities,  activities  which 
can  be  continually  adjusted  as  new  conditions  arise. 

The  notion  of  instructional  design  as  an  activity  with  many  component  processes 
which  are  completed  in  an  iterative,  overlapping  or  even  concurrent  fashion  is  not 
inconsistent  with  the  original  presentations  of        J  systems  theory,  Systems  were  viewed 
as  existing  within  and  interacting  with  an  environment.  Systems  weru  viewed  as  a  whole. 
Systems  were  viewed  as  dynamic.  Currently,  these  attributes  are  being  reexamined  in 
terms  of  systemic  design,  as  opposed  to  systematic  design.  The  use  of  the  word  "systematic" 
implies  using  specified  design  procedures,  while  the  word  "systemic"  implicH  concurrent 
consideration  of  the  many  aspects  of  a  situation  which  can  affect  the  learning  procimn 
(Richey,  1992).  There  is  more  emphasis  on  design  as  creative  problem  solving 
(Romiszowski,  1981),  flexibility,  and  adaptability  in  systemic  design,  Not  only  dues  the 
systemic  approach  imply  the  consideration  of  a  wider  range  of  variable  in  the  t\vnW\ 
process,  it  also  implies  the  use  of  a  nonlinear  orientation.  It  prevents  the  use  of  piecemeal 
solutions  to  education  and  training  problems. 

The  Role  of  Analysis  in  Design  Models  of  the  Future.  The  role  find  importance  of 
instructional  context  has  been  expanding,  bolstered  not  only  by  the  theoretical  emphasis  on 
environment  in  systems  theory,  but  also  by  the  current  impact  of  constructivism,  cognitive 
Psychology,  and  performance  technology.  While  such  theory  also  iH  influencing  the 
uevelopment  of  particular  analysis  techniques  (such  as  cognitive  task  annlyHiH),  the 
emphasis  here  is  upon  anticipated  changes  in  the  ISD  models  themselves,  especially  with 
respect  to  the  various  forms  of  front-end  analysis. 

Seels  (1989)  indicates  that  the  "const rue tivist  paradigm  state*  that  learning  occurs 
because  personal  knowledge  is  constructed  by  an  active  and  self-regulated  learner  who 
resolves  conflicts  between  'eas  and  reflects  on  theoretical  explanations"  (p.  lit), 
Constructivism  further  holds  that  meaning  is  based  upon  experience  and  the  context  in 
which  that  experience  takes  place  (Duffy  and  Jonassen,  1991).  The  construct  ivist  influence 
is  permeating  much  thinking  on  instructional  design,  and  is  part  of  the  growing  emphasis  on 
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the  importance  of  learner  characteristics  in  the  design  task.  Rather  than  concentrating 
primarily  upon  the  learner's  prerequisite  skills,  learner's  experiential  background,  attitudes, 
and  interests  are  also  being  uddressed.  Such  considerations  become  one  feature  of  the 
newer,  expanded  needs  assessment  orientation. 

In  addition  to  this  broader  consideration  of  learner  characteristics,  new  pre-design 
analysis  procedures  are  also  breaking  away  from  the  old  almost  total  emphasis  on  subject 
matter  by  addressing  the  nature  of  the  environment  as  well.  For  example,  Tessmer  and 
Harris  (1992)  proposed  an  analysis  which  addressed  the  learning  environment  and  the 
support  environment.  Richey  (1992)  has  demonstrated  the  critical  role  of  not  only  a  variety 
of  learner  characteristics,  but  the  learner's  perceptions  of  the  organizational  climate  in 
training  design.  Richey  and  Tessmer  (in  press)  have  now  merged  these  two  approaches  in  a 
comprehensive  model  of  contextual  analysis  which  includes  consideration  of  three  temporal 
environments  in  the  front-end  analyses:  pre-instructional,  instructional,  and  post- 
instructional.  Each  environment  is  analyzed  to  determine  the  essential  physical  and  psycho- 
social factors  from  the  immediate  and  surrounding  environment  that  influence  learning  and 
transfer  of  training. 

Contextual  analysis  is  also  prompted  by  the  increasingly  wide-spread  influence  of 
performance  technology.  Here,  the  orientation  is  toward  solving  organizational  performance 
problems,  with  education  and  training  seen  as  only  one  possible  solution.  The 
characteristics  of  the  environment  are  critical  during  the  front-end  analysis  which  is  viewed 
more  of  a  problem  analysis  directed  toward  the  larger  organizational  setting,  as  opposed  to 
a  needs  assessment  directed  primarily  toward  learner  knowledge  and  skill  deficiencies. 

The  Role  of  Evaluation  in  the  Design  Models  of  the  Future.  Interestingly,  most 
current  ISD  models  do  not  include  a  comprehensive  notion  of  evaluation.  The  majority  of 
models  reviewed  by  Andrews  and  Goodson  (1991)  addressed  only  formative  evaluation  of 
products  and  programs  and  needs  assessment,  but  did  not  specifically  include  a  summative 
evaluation  phase.  Notable  exceptions  are  the  design  models  of  Dick  and  Carey  (1990)  and 
Seels  and  Glasgow  (1990).  It  is  anticipated  that  not  only  will  the  models  of  the  twenty-first 
century  address  both  formative  and  summative  evaluation,  but  that  the  nature  of  such 
evaluation  will  be  expanded  from  what  typically  occurs  today.  These  new  approaches  will 
occur  to  some  extent  by  those  concerns  with  context  that  are  now  beginning  to  impact  the 
analysis  stage  of  design. 

These  context  considerations  have  been  generated  by  the  conceptual  influence  of  both 
performance  technology  and  the  total  quality  management  (TQM)  movements.  Essentially, 
both  movements  are  concerned  with  the  corporate  "bottom  line"  and  satisfying  the 
requirements  of  the  customer,  and  such  emphases  negate  traditional  assessment  and 
evaluation  procedures  which  are  limited  to  examining  short-term  learner  outcomes.  Instead, 
they  stress  what  Kirkpatrick  (1983)  has  previously  labeled  as  Level  3  and  4  evaluations. 
Here  assessments  are  made  of  the  application  of  knowledge  and  skills  and  upon  subsequent 
organizational  impact,  rather  than  learner  reactions  to  instruction  and  knowledge 
acquisition.  Level  3  and  Level  4  summative  evaluations  are  likely  to  be  demanded, 
especially  in  the  workplace,  but  it  is  not  unlikely  that  school  environments  will  also  require 
such  data.  Moreover,  it  is  being  suggested  that  designers  also  use  Level  3  and  Level  4 
techniques  when  conducting  formative  evaluations  as  well  as  summative  (Dick  and  King,  in 
press). 

The  concerns  with  retention  and  transfer  of  training  also  are  evident  in  the  addition 
of  confirmative  evaluation  processes  to  a  typical  ISD  model  as  has  been  suggested  by 
Hellebrandt  and  Russell  (1993).   The  object  of  this  endeavor  is  to  determine  the  continuing 
competence  of  learners  or  the  continuing  effectiveness  of  instructional  materials.  Not 
unrelated  to  Level  3  evaluation,  confirmative  evaluation  complements  the  thrusts  of 
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situated  cognition  and  anchored  learning  which  address  the  importance  of  embedding 
learning  activities  in  realistic  contexts  as  a  means  nf  promoting  transfer  of  training  (Streibel, 
1991;. 

\lu  ro-Desimi  Models  of  the  Year  2000 

In  addition  to  tho**e  enhancements  of  the  typical  macro  design  model,  micro- models 
which  guide  the  selection  and  sequencing  of  instructional  strategies  art1  also  likely  m  change 
in  the  twenty  first  century.  Two  key  micro-model  issues  which  have  been  stimulated  by 
emerging  theoretical  developments  will  he  discussed  here      the  nature  of  learner  control, 
and  the  transfer  of  training. 

The  Nature  of  Learner  Control  in  Design  Models  of  the  Future.  At  the  heart  of 
constructivism  is  the  issue  of  learner  control.  The  true  const  ructivist  believes  that  meaning 
and  reality  are  functions  of  individual  interpretation  (Jonassen,  1991;  Lebow,  1993).  An 
extension  of  this  orientation  to  instructional  design  then  implies  that: 

instructional  goals  and  objectives  would  be  negotiated  rather  than  imposed 
upon  the  learner; 

sequences  cf  learning  activities  would  be  flexible  rather  than  prescribed; 
learning  strategies  would  not  be  dictated  by  the  design;  and 
evaluation  would  be  less  criterion  referenced  (Jonassen,  1991). 

Aspects  of  this  orientation  are  not  entirely  unrelated  to  the  conclusions  of  both 
cognitive-based  designers  (Wilson  and  Cole,  1991),  as  well  as  to  some  who  have  struggled  to 
develop  instruction  using  the  full  capabilities  of  the  new  technologies.  For  example, 
1  lannafin  ( 19**2)  speaks  of  the  possibilities  the  interactive  technologies  offer  for  student- 
i  entered  learning.  In  these  situations,  the  designs  "focused  on  supporting  student-initiated 
lesson  navigation,  providing  an  organizing  theme  or  context  for  lesson  activities,  and 
embedding  aids  and  support  in  the  form  of  help,  elaboration,  and  other  resources  that  ca^ 
be  selected  by  the  student  to  improve  understanding  (Hannafin,  1992t  p.  54). 

Such  environments  are  far  less  structured  thai:  i.  tvpical  of  those  designed  using 
systematic  procedures  on  hoth  a  macro-  and  micro. Ipvi  :.  \Vhile  this  is  compatible  with  th" 
newer  technology-based  instruction,  fciich  as  a  hypertext  lesson,  it  is  not  nearly  so 
compatible  with  environments  demanding  accountahiii';*.  Fo/  example,  the  TQM  move  a.  at 
is  predicated  upon  organizational  objectives  (i.e.  a  vision  cr  mission  statement),  policies,  and 
procedures  that  specify  the  manner  in  which  the  objective  will  be  achieved  (Caplan,  1993). 
Likewise,  performance  technology  relies  upon  a  high  degree  of  structure  to  meet,  evaluate, 
and  report  the  impact  of  instruction  upon  the  solution  of  organizational  problems.  Such 
orientations  are  not  alien  to  many  instructional  designers  who  adhere  to  traditional  systems 
thinking  (Dick,  1993b).  but  they  do  not  seem  to  facilitate  the  types  of  learning  environments 
proposed  by  I  lannafin,  Jonassen.  or  Lebow. 

This  philosophical  dichotomy  transcends  other  aspects  of  micro-design.  Another  key 
example  ix  that  of  message  design.  Berry  (in  press)  describes  the  radial  shifts  in  future 
message  design,  stimulated  by  learner  centered  design  and  situated  cognition,  in  which 
instructional  environments  will  facilitate  learners  as  they  explore  areas  of  knowledge  and 
develop  their  own  versions  of  that  knowledge.  "The  development  of  such  systems  is 
predicated  on  a  shift  from  design  of 'stimulus  messages'  to  interface  dergn"  (Berry,  in  press, 
p.  8).  This  is  a  radical  departure  for  traditional  T.ossage  design,  and  the  dime  .si'  s  of  such 
change  arc  compounded  by  the  implications  of  the  new  technologies,  technologies  *nat.  not 
only  provide  for  nearly  full  fidelity  and  a  wider  arrav  of  formats,  but  also  for  the  design  ot 
message  environments  and  experiences. 

Transfer  of  Training  Strategies  in  Design  Models  of  the  Future.  Transfer  of  training 
is  critical  in  most  education  and  training  environments,  and  to  many,  serves  as  a  criterion 


for  effective  instruction.  Consequently,  it  serves  as  a  logical  focus  of  analyses  of  strategy 
selection  and  sequencing.  Current  transfer  discussions  have  emerged  to  a  great  extent  from 
an  examination  of  situated  learning  and  its  applications  in  instructional  design.  The  issue 
centers  on  the  relative  effectiveness  of  using  either  situated,  authentic  tasK^  or  generalized 
principles  in  instruction  as  a  means  of  promoting  transfer.  Winn  (1993)  "rgues  that 
traditional  instructional  design  (ID)  and  situated  learning  (SL)  jjre  compatible  if 
compromises  are  made: 

For  ID  this  mesns  primarily  re-integrating  design  with  instructional  delivery 
so  that  the  interaction  between  student  and  instruction  becomes  more 
adaptive  to  situations  that  vary  from  one  problem  to  the  next.  It  also  means 
the  development  of  learning  environments  in  which  students  construct 
knowledge  for  themselves.  For  their  part,  proponents  of  SL  need  to  develop 
effective  ways  for  bringing  authentic  activity  into  the  classr.      or  for  better 
articulating  schooi-based  and  community-based  activity  (p.  J). 
These  compromises  highlight  the  instructional  strategy  changes  that  are  suggested 
by  situated  learning,  changes  which  alter  the  common  use  of  generic  prescriptions.  While 
these  changes  are  not  inconsistent  with  the  recommendations  which  have  been  made  in 
other  quarters  wit!:  respect  to  learner  control,  they  are  not  uniformly  accepted.  Tripp  (1993), 
for  example,  continues  to  question  the  ability  of  "situated"  instructional  activities  to  promote 
transfor.  For  example,  while  supporting  the  importance  of  showing  the  relevance  of  content, 
he  cites  the  barriers  created  for  many  learners  by  the  complexities  of  the  "real  world"  when 
they  are  introduced  into  instruction. 

Other  lines  of  current  thinking  also  provide  direction  for  promoting  transfer.  The 
emphasis  upon  contextual  variables  in  constructivism  and  performance  technology  also 
speaks  to  transfer  of  training.  Richey  and  TesHruer's  (in  press)  contextual  analysis  model 
speaks  to  environmental  cues  and  incentives  which  impact  transfer,  as  well  as  opportunities 
for  content  use  and  application  in  diverse  settings.  Such  data  suggests  modifications  in  both 
learning  activities  and  supposing  instructional  management  systems. 

Summary 

The  stimulation  currently  provided  by  the  rich  intellectual  activity  in  the  realm  of 
instructional  design  is  creating  the  likelihood  of  change,  not  only  in  the  generic  procedural 
models  traditionally  used  to  guide  !SD  projects,  but  also  in  the  types  of  the  strategies  used 
to  achieve  learning  goals.  Some  of  these  changes  are  obviou»  row;  others  will  become  more 
common  as  we  approach  the  new  century.  As  the  Assign  generations  evolve,  it  is  possible 
that  the  high  degree  of  procedural  unanimity  the  field  has  previously  enjoyed  may  be 
replaced  by  more  diversity.  For  example,  there  may  be  macro- design  models  which  do  not 
advocate  the  specification  of  goals  and  objective  There  may  be  micro-design  models  which 
do  not  advocate  sequencing  using  the  tradition?  :  ierarchical  approach  based  upon  Gagne's 
Cumulative  Learning  Theory.  This  diversity,  vhile  admittedly  unsettling  to  many  of  us,  is 
also  exciting,  and  offers  a  promise  of  not  only  new  models,  bu'  new  challenges  to 
practitioners  and  researchers  alike. 
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Developmental  KeMiwitch:  The  Definition  and  Scope 


The  field  of  Instructional  Technology  has  traditipnally  involved  a  unique  blend  of 
theory  and  practice.  This  blend  is  most  obvious  in  developmental  research  which  involves 
the  production  of  knowledge  based  upon  situation-specific  problem  solving.  Developmental 
research,  as  opposed  to  simple  instructional  development,  has  been  defined  as  "the 
systematic  study'of  designing,  developing  and  evaluating  instructional  programs,  processes 
and  products  that  must  meet  the  criteria  of  internal  consistency  and  effectiveness"  (Seels 
and  Richey,  in  press).  In  its  simplest  form,  developmental  research  could  be  either: 
a  situation  in  which  someone  is  performing  instructional  design, 
development,  or  evaluation  activities  and  studying  the  process  at  the 
same  time;  or 

the  study  of  the  impact  of  someone  else's  instructional  design  and 
development  efforts;  or 

the  study  of  the  instructional  design,  development,  and  evaluation 
process  as  a  whole,  or  of  particular  process  components. 
In  each  case  the  distinction  is  made  between  performing  a  process  and  studying  that 
process.  Reports  of  developmental  research  may  take  the  form  of  a  case  study,  an 
evaluation  report,  a  retrospective  analysis,  or  even  that  of  a  typical  experimental  research 
report.  Today,  even  amid  the  calls  for  increased  use  of  alternative  research  methodologies, 
the  notion  of  instructional  development  as  a  research  methodology  is  unclear,  not  only  to  the 
broader  community  of  educational  researchers,  but  to  many  Instructional  Technology 
researchers  as  wen. 


The  Background  of  Developmental  Research 


The  field  of  Instructional  Technology  as  it  exists  today  emerged  from  a  convergence  of 
the  fields  of  audiovisual  education  and  instructional  psychology.  In  audiovisual  education 
the  emphasis  was  upon  the  role  of  media  as  an  enhancement  of  the  teaching/learning 
process  and  an  aid  in  the  communication  process,  and  there  was  much  interest  in  materials 
production.  On  the  other  hand,  in  instructional  psychology  the  nature  of  the  learner  and  the 
learning  process  took  precedence  over  the  nature  of  the  delivery  methodology,  and  there  was 
much  interest  in  instructional  design.  Complementing  the  instructional  psychology  roots 
was  the  application  of  systems  theory  to  instruction  which  resulted  in  the  instructional 
systems  design  movement  (Seels  and  Richey,  in  press).  This  conceptual  and  professional 
merger  came  to  fruition  in  the  1960's  and  1970's.  During  this  period  instructional  design 
and  development  came  to  assume  the  role  \  the  "linking  science"  that  John  Dewey  had 
called  for  at  the  turn  of  the  century  (Reigeluth,  1983). 

Not  surprisingly,  it  was  during  this  same  period  that  development  emerged  as  a 
research  endeavor.  This  change  was  exemplified  by  the  shift  in  topics  between  the  First  and 


'This  paper  is  excerpted  from  "Developmental  Research"  a  chapter  by  Rita  Richey.  Philip  Doughty,  and 
Wayne  Nelson  which  will  be  published  in  the  forthcoming  Handbook  of  Research  on  Educational 
Communications  and  IttdmiW* edited  by  David  H.  Jonassen. 


Second  Handbooks  of  Hoaearch  on  Teaching  (Gage,  1963;  Travers,  1973).  In  the  1963 
handbook,  media  wan  iiddrontu'd  iih  an  area  of  research,  and  all  research  methodologies 
considered  were  quantitative.  In  the*  1l)7;i  handbook,  media  continued  to  be  included  as  a 
research  area,  but  the  research  methodologies  were  varied,  including  Eva  Baker's  chapter  on 
"The  Technology  of  Instructional  Development".  This  chapter  describes  in  detail  the  process 
of  systematic  product  design,  development,  and  evaluation.  Of  significance  is  the  fact  that 
the  entire  methodology  motion  was  titled  "Methods  and  Techniques  of  Research  and 
Development". 

This  was  a  period  in  which  federal  support  of  educational  research  mushroomed. 
Regional  research  and  development  laboratories  were  established  and  the  ERIC  system  was 
devised  for  dissemination.  Clifford  (1973)  estimated  that  appropriations  for  educational 
"research  and  development  for  1966  through  1968  alone  equaled  three-fourths  of  all  funds 
ever  made  available"  (p.  1).  Research -based  product  and  program  development  had  become 
firmly  established  as  part  of  the  scientific  movement  in  education.  At  this  time,  Wittrock 
(1967)  hailed  the  use  of  empirical  measurement  and  experimentation  to  explain  product 
effectiveness.  Such  activities  "could  change  the  development  of  products  into  research  with 
empirical  results  and  theory  generalizable  to  new  problems"  (p.  148). 

Hilgard  (1964)  characterized  research  as  a  continuum  from  basic  research  on  topics 
not  directly  relevant  to  learning  through  the  advocacy  and  adoption  stages  of  technological 
development.  Saettler  (1990)  maintained  that  the  last  three  of  Hilgard's  research  categoi.es 
were  directly  within  the  domain  of  Instructional  Technology.  These  included  laboratory, 
classroom  and  special  teacher  research;  tryout  in  "normal"  classrooms;  and  advocacy  and 
adoption.  Note  that  these  are  portrayed  as  types  of  research,  rather  than  applications  of 
research,  and  they  are  all  encompassed  within  the  framework  of  developmental  research. 

While  Instructional  Technology  is  not  the  only  field  concerned  with  learning  in 
applied  settings,  few  would  dispute  the  critical  role  played  by  these  three  types  of  research 
in  our  field.  Moreover,  our  uniqueness  among  educational  fields  is  not  only  our  concern  with 
technology,  but  rather  our  emphasis  upon  the  design,  development,  and  use  of  processes  and 
resources  for  learning  (Seels  and  Richey,  in  press).  Given  this  definition  of  the  field, 
developmental  research  is  critically  important  to  the  evolution  of  the  field's  theory  base. 


The  Character  of  Developmental  Research 

The  distinctions  between  "doing"  and  "studying"  design  and  development  provide 
further  clarification  of  development  as  a  research  activity.  These  distinctions  can  be 
described  in  terms  of  examining  the  focus,  techniques,  and  tools  of  developmental  research. 

The  Focus  of  Developmental  Research.   The  general  purposes  of  research  have  been 
described  as  knowledge  production,  understanding  and  prediction.  Within  this  framework, 
developmental  research  has  particular  emphases  which  vary  in  terms  of  the  extent  to  which 
the  conclusions  are  generalizable  or  contextually-specific.  Figure  1  portrays  the  relationships 
among  the  various  types  of  developmental  research. 

The  most  common  developmental  research  projects  fall  into  the  first  category  of 
Figure  1.  This  category  typically  involves  situations  in  which  the  product  development 
process  is  analyzed,  described,  and  the  final  product  is  evaluated,  such  as  Buch's  (1987) 
documentation  of  the  development  of  an  industrial  microcomputer  training  program.  Driscoll 
(1991)  called  this  research  paradigm  systems-based  evaluation.  Some  Type  1 
developmental  studies  reflect  traditional  evaluation  orientations  in  which  the  development 
process  is  not  addressed,  and  only  the  product  or  program  evaluation  is  described.  An 
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example  of  this  type  of  study  is  O'Quin,  Kin*f%  und  HeeryV  (1987)  report  of  the  evaluation 
of  a  micro-computer  training  workshop  for  college  personnel.  Regardless  of  the  nature  of  the 
Type  1  study,  the  results  are  typically  context  and  product  specific,  even  though  the 
implications  for  similar  situations  may  be  discussed. 


FIGURE  1 
A  Summary  of  the  Types 
of  Developmental  Research 


DEVELOPMENTAL  RESEARCH 


Type  1 


Type  2 


Type  3 


Description  or  Analysis  of 
Product  or  Program  Design, 
Development  &  Evaluation 


Description  or  Analysis  of 
Product  or  Program 
Utilization  &  Impact 
Evaluation 


Study  or  Improvement  of 
the  Design,  Development  & 
Evaluation  Process  or 
Components . 


(Study  of  Specific 
Products  or  Contexts) 


(Study  of  Extended  Impact 
of  Product  or  Program  on 
Organization  Change  &/or 
Learner  Growth) 


(Design,  Development,  & 
Evaluation  Procedural 
Model  Development) 


Context-Specific 
Conclusions 


Generalized 
Conclusions 


The  focus  of  Type  2  developmental  research  is  not  so  much  upon  the  development  of 
products,  but  more  upon  the  impact  of  that  product  upon  the  learner  or  the  Figure  1 
organization.  Using  Kirkpatrick's  (1983)  categories  of  evaluation  outcomes,  these  studies 
typically  do  not  address  only  participant  satisfaction  or  evidence  of  learning,  but  tend  to 
consider  behavior  or  skill  application  and  organizational  impact.  They  address  product  or 
program  effectiveness  on  a  broader  scale,  such  as  Smith's  (1993)  comprehensive  study  of  five 
years  of  executive  development.  Type  2  studies  may  employ  confirmative  evaluation 
procedures.  This  typicallv  involves  an  evaluation  as  a  part  of  a  summative  study  and  is 
designed  to  determine  the  continuing  competence  of  learners  or  the  continuing  effectiveness 
of  instructional  materials  (Hellebrandt  and  Russell,  1993).   Type  2  studies  also  attend  to 
program  implementation  and  maintenance  issues.  One  example,  of  this  latter  focus  is  tin* 
case  study  of  cost-effectiveness  evaluation  described  by  Klein  and  Doughty  (1980).  Their 
conclusions,  as  is  typical  of  a  Level  2  study,  are  context- specific,  but  they  also  suggest 
general  principles  for  cost -effectiveness  evaluation  procedures  which  can  he  used  in  a  variety 
of  design  and  development  projects.  In  this  respect,  the  study  approaches  the  purposes  nf  a 
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Type  11  developmental  research  project. 

Tin*  third  typo  of  study  is  oriented  toward  a  general  analysis  of  cither  design 
development  or  evaluation  processes  as  a  whole  or  any  particular  component.  They  arc 
similar  to  those1  studios  Driscoll  (1991)  calls  model  development  mid  technique  development 
research,  While  there  an*  fewer  studies  that  focus  on  the  more  global  orientation,  Taylor 
and  Ellis's  (1991)  study  did  no  by  evaluating  the  use  of  instructional  systems  design  in  the1 
Navy  and  Kress  (1987)  did  so  by  comparing  the  impact  of  systematically  designed  training 
with  a  non-systematic  approach.  Other  studies  in  this  category  focus  upon  only  one  phase  of 
the  design/development/evaluation  process,  such  as  Jonassen's  (19HH)  case  study  of  using 
needs  assessment  data  in  the  development  of  a  university  program.  Type  'A  research  may 
draw  its  population  from  either  one  target 

project  such  as  King  and  Dille's  (1993)  study  of  the  application  of  quality  concepts  in  tin1 
systematic  design  of  instruction  at  the  Motorola  Training  and  Education  Center,  or  from  a 
variety  of  design  and  development  environments.  Examples  of  the  latter  approach  include 
Kiplinger's  (1985)  survey  of  current  task  analysis  procedures,  and  Cambre's  (I97H)  historical 
study  of  formative  evaluation  in  instructional  film  and  television.  Typically,  conclusions  from 
Type* ,')  developmental  research  are  generalized,  even  though  there  arc  instances  of  context 
specific  conclusions  in  the  literature. 

Nun- developmental  Research  in  the  Field.  A  critical  aspect  of  any  concept  definition 
is  the  identification  of  non-examples  as  well  as  examples.  This  is  especially  important  with 
respect  to  developmental  research  since  it  often  seems  to  overlap  with  other  key 
methodologies  used  in  the  field.  Even  so,  developmental  research  does  not  encompass 
studies  such  as  the  following: 

instructional  psychology  studies; 

media  or  delivery  system  comparison  or  impact  studies; 

message  design  and  communication  studies; 

policy  analysis  or  formation  studies;  and 

research  on  the  profession. 
While  results  from  research  in  these  areas  impact  the  development  process,  the 
study  of  these  variables  does  not  constitute  developmental  research.  For  example,  design 
and  development  is  dependent  upon  what  we  know  about  the  learning  process.  We  have 
learned  from  the  research  literature  that  transfer  of  training  is  impacted  by  motivation, 
organizational  climate,  and  previous  educational  experiences.  Therefore,  one  may  expand  a 
front -(Mid  analysis  to  address  such  issues,  or  even  construct  design  models  which  reflect  this 
information,  hut  the  foundational  research  would  not  be  considered  developmental,  If  the 
new  models  were  tested,  or  programs  evaluated  which  were  designed  using  such  models, 
this  research  would  qualify  as  developmental. 

A  fundamental  distinction  should  be  made  between  repoits  of  actual  developmental 
research,  and  descriptions  of  design  and  development  procedural  models.  While  these 
models  may  represent  a  synthesis  of  the  research,  they  do  not  constitute  research  in 
themselves.  A  good  example  of  this  latter  situation  is  Park  and  Hannafin's  (1993) 
guidelines  for  designing  interactive  multimedia.  These  guidelines  are  generalized  principles 
which  speak  to  the  development  process,  and  they  are  based  upon  a  large  body  of  research. 
Nonetheless,  the  identification  and  explanation  of  the  guidelines  is  not  in  itself  an  example 
of  developmental  research.  The  Instructional  Technology  literature  includes  many  examples 
of  such  work.  They  often  provide  the  stimulus  for  a  line  of  new  research,  even  though  these 
articles  themselves  are  not  considered  to  be  research  reports  themselves.  There  are  many 
examples  today  of  such  work,  including  explorations  of  topics  such  as  cognitive  task  analysis 
(Ryder  and  Redding,  199U),  or  the  nature  of  design  and  designer  decision  making  (Rowland, 


The  Techniques  and  Tools  of  Developmental  Research.  Developmental  researchers 
employ  a  variety  of  research  methodologies,  applying  any  tool  which  meets  their 
requirements.  Sunimative  evaluation  studies  often  employ  classical  experimental  designs. 
Needs  assessments  may  incorporate  qualitative  approaches.  Process  studies  may  adopt 
descriptive  survey  methods.  Even  historical  research  methods  may  be  used  in 
developmental  projects. 

Traditional  research  tools  and  traditional  design  tools  facilitate  the  developmental 
endeavor.  Expertise  is  often  required  in  statistical  analysis,  measurement  theory,  and 
methods  of  establishing  internal  and  external  validity.  Likewise,  the  developmental 
researcher  (even  those  studying  previously  designed  instruction)  requires  a  command  of 
design  techniques  and  theory.  Additional  design  proficiency  is  frequently  required  when 
using  electronic  design  systems  and  aids,  conducting  environmental  analyses,  and  defining 
ways  to  decrease  design  cycle  time. 

A  developmental  research  project  may  include  several  distinct  stages,  each  of  which 
involves  reporting  and  analyzing  a  data  set.  Merely  conducting  a  comprehensive  design  and 
development  project  does  not  constitute  conducting  a  developmental  research  project  even 
using  its  most  narrow  Type  1  definition.  One  must  also  include  the  analysis  and  reporting 
stage  to  warrant  being  classified  as  developmental  research. 

Developmental  research  projects  may  include  a  number  of  component  parts.  Sub- 
studies  may  be  conducted  to  analyze  and  define  the  instructional  problem,  to  specify  the 
content,  or  to  determine  instrument  reliability  and  validity.  Sub-studies  may  be  conducted 
to  provide  a  formative  evaluation,  a  summative  evaluation,  or  a  follow-up  of  post -instruction 
performance.  Consequently,  reports  of  developmental  research  are  frequently  quite  long, 
often  prohibiting  publication  of  the  full  study. 

Reports  of  developmental  projects  can  often  be  found  in: 
professional  journals; 
doctoral  dissertations; 

Educational  Resource  Information  Center  (ERIC)  collections  of 

unpublished  project  reports;. and 

conference  proceedings. 
The  nature  of  the  reports  vary  depending  upon  the  dissemination  vehicle.  Sometimes,  full 
developmental  projects  are  split  into  more  easily  publishable  units  (or  even  summarized)  to 
facilitate  publication  in  the  traditional  research  journals.  Developmental  research  reports 
are  also  published  in  practitioner-oriented  journals  and  magazines,  and  the  methodology 
and  theoretical  base  of  the  studies  is  omitted  to  conform  to  the  traditions  of  those 
periodicals. 

Conclusion 

Developmental  research,  even  though  frequently  misunderstood,  has  contributed 
much  to  the  development  of  the  field  as  a  whole.  It  often  serves  as  a  vehicle  for 
dissemination  of  model  techniques  and  processes,  especially  as  new  technologies,  new 
procedural  changes,  new  programmatic  trends  emerge.  Moreover,  developmental  research 
can  provide  n  basis  for  both  model  construction  and  theorizing,  one  which  is  rooted  in  the 
experiences  of  practitioners  as  well  as  researchers. 
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Abstract 

The  present  study  wan  a  fcrmative  evaluation  of  the  The  Intelligent  Physics  Tutor,  an 
interactive  Hoftwure  program  designed  to  improve  physics  students1  problem-solving  skills. 
Multiple  data  sources  used  in  the  evaluation  included  teacher  experiences  and  attitudes; 
and  student  attitudes,  achievement,  and  problem-solving  strategies.  Findings  from  a 
semester-long  tryout  in  a  high  school  physics  class  indicated  that  both  the  teacher  and  the 
students  were  very  positive  about  the  Tutor's  instructional  benefits.  The  most  favorable 
reactions  were  by  lower-ability  students  who  had  trouble  solving  problems  on  their  own. 
While  there  were  no  differences  in  achievement  between  the  Tutor  class  and  two  control 
classes,  Tutor  students  demonstrated  relatively  greater  use  of  systematic  problem-solving 
strategies  over  time. 


Introduction 

As  orientations 

in  instructional  theory  and  practice  continue  to  evJvc  from  their  behavioral  roots,  cognitive 
and  "constructivist"  viewpoints  have  had  increasing  influence  on  the  design  of  computer- 
based  instruction  (CBI)  (Bednar,  Cunningham,  Duffy,  &  Ferry,  1991;  Di  Vesta  &  Rieber, 
1987).  Such  viewpoints  emphasize  the  processes  by  which  students  learn  higher-order  skills 
and  the  attitudes  and  interests  they  develop  through  their  experiences  The  marriage  today 
of  cognitive  perspectives  with  interactive,  automated  delivery  systems  orients  today's 
designers  toward  considering  how  learner  needs  and  mental  models  can  be  diagnosed  on 
task  and  instruction  adapted  accordingly.  The  present  project,  involving  the  design  and 
evaluation  of  an  intelligent  tutoring  system  for  high  school  physics,  was  grounded  on  this 
orientation. 

The  Intelligent  Physics  Tutor  Project 

The  Intelligent 

Physics  Tutor  project  (Loftin,  1990),  was  sponsored  by  NASA/Johnson  Space  Center  and 
Apple  Classrooms  of  Tomorrow.  Its  purpose  is  to  create  an  interactive  environment,  using 
the  architecture  of  an  intelligent  tutoring  system,  in  which  physics  students  can  practice 
problem-solving  applications  while  receiving  individualized  guidance  and  feedback. 
Consequently,  as  students  work  on  the  Tutor  in  class,  the  teacher  is  freed  to  give  special 
attention  to  individuals  who  need  it.  The  Tutor  operates  on  an  Apple  Macintosh  II 
computer  and  makes  use  of  high  resolution  graphics  and  sound. 

The  basic  instructional  model  is  designed  to  diagnose  students1  misconceptions  while 
they  solve  problems.  The  identified  misconceptions  then  form  the  basis  for  determining  the 
immediate  feedback  provided  on  the  current  problem  and  for  selecting  additional  problems 
to  he  worked.  The  analysis  takes  place  via  a  "local  strategy"  that  records  every  response  (as 
"student  action  codes")  that  the  student  makes  in  solving  the  problem.  Using  the  work 
space,  an  available  calculator,  the  keyboard,  and  given  variables  and  equations  that  can  be 
selected  for  the  problem  solution,  the  student  attempts  to  work  the  problem.  At  each  step, 
a  comparison  is  made  between  the  student's  action  and  the  correct  or  expected  action.  If  a 
discrepancy  is  indicated,  the  most  likely  misconception  for  that  problem  is  diagnosed  on  the 
basis  of  the  error  response  and  the  cumulative  action  codes.  Appropriate  feedback,  as 
outlined  below,  is  then  provided. 

Students  receive  on-task  feedback  messages  only  when  an  error  is  detected; 
otherwise  they  are  allowed  to  proceed  without  interruption.  An  example  of  an  error  feedback 
display  is  shown  in  Figure  1.  The  instructor  pictured  is  Dr.  Howen  I/jftin,  the  Tutor 
developer;  individual  teachers  can  scan  their  own  pictures  into  the  program  to  personalize 
this  feedback. 

The  error  messages  provided  are  geared  to  the  specific  misconception  diagnosed  and 
become  increasingly  specific*  as  errors  reflecting  that  misconception  accumulate.  Importantly, 
by  correcting  the  error  at  the  time  it  is  made,  the  system  prevents  the  learner  from 
proceeding  down  an  incorrect  pathway  (and  thus  losing  time  and  experiencing  frustration). 
Hased  on  the  cumulative  frequencies  of  misconceptions  recorded  for  the1  individual  across 
completed  problems,  the  Tutor's  "global  model"  then  selects  a  new  problem  that  requires 
practicing  skills  directly  related  to  the  diagnosed  areas  of  weaknesses  (i.e.,  the  specific 
misconceptions  having  the  highest  frequencies  of  incurrences). 

Additional 

feedback  systems  provide  summaries  and  analyses  of  performances  for  both  classes  (we 
Figure  2)  and  individuals  (see  Figure  U).  The  basic  feedback  display  shows  the 


misconceptions  currently  diagnosed  while  suggesting  a  prescription  of  how  many  related 
exercises  (or  problems)  should  be  given  for  strengthening  the  problem-solving  skills  for  a 
given  misconception.   Parallel  feedback  for  the  class  as  a  whole  provides  useful  guidance  for 
whole-group  followup  instruction. 
Evaluation  Objectives 

The  purposes 

of  the  present  evaluation  were  threefold.  One  was  to  describe  how  the  Tutor  was  being  used 
in  the  pilot  high  school  class.  This  information  was  considered  important  for  (a)  identifying 
possible  strategies  for  integrating  the  Tutor  with  conventional  instructional  methods,  and  (b) 
understanding  the  application  context  in  which  results  occurred.  A  second  purpose  was  to 
determine  student  and  teacher  reactions  to  the  Tutor  with  regard  to  weaknesses,  strengths, 
and  suggestions  for  improvement.  Based  on  this  information,  more  confident  decisions  can 
be  made  about  needs  for  "fine-tuning"  the  user  interface  and  program  content.  A  third 
purpose  was  to  examine  possible  impacts  of  using  the  Tutor  on  problem-solving  strategy  and 
performance.  This  interest  included  both  the  processes  (how  a  problem  was  solved)  and 
products  (accuracy  of  solutions)  of  problem-solving  activities. 

In  interpreting 

the  results,  readers  should  be  aware  of  several  limitations  of  the  present  evaluation  context. 
First,  the  population  tested  consisted  of  honors  physics  students  attending  a  middle-class 
suburban  high  school.  These  students  were  more  capable  and  experienced  problem  solvers 
than  are  typical,  beginning  physics  students  at  moat  schools  in  the  United  States.  Second, 
as  will  be  described  below,  the  inability  to  randomly  assign  students  to  "treatment"  and 
control  classes  necessitated  the  use  of  a  "quasi -experimental"  design,  i.e.,  comparison  of  pre- 
existing classes.  Third,  only  one  teacher  was  involved,  thus  confounding  Tutor  outcomes 
with  her  personality,  orientation,  and  style.  Given  that  these  factors,  the  present  evaluation 
was  designed  to  be  primarily  "formative"  in  nature,  in  the  sense  of  identifying  how  the  Tutor 
operates  and  its  "likely"  effects.  A  summative  evaluation  (testing  its  "effectiveness")  would 
be  the  next  logical  step  in  the  research  and  development  process. 
Evaluation  Methodology 
Participants  and  Design 

A  quasi- 
experimental  design  consisting  of  three  intact  classes  of  students  was  employed.  All  were 
honors  physics  classes  taught  by  Ms.  Beverly  Lee  at  Clear  Creek  High  School. 
Unfortunately,  due  to  chance  factors,  the  three*  classes  differed  somewhat  in  size  and  in 
ability.  It  was  therefore  decided  to  provide  11  strong  rather  than  weak  test  of  the  Physics 
Tutor  though  the  selection  of  the  lowest-ability  (and  second  InrgcHt)  class  as  the  PT  group. 
This  class  was  taught  sixth  period  and  consisted  of  1IJ  students.  The  PT  group  worked 
problems  on  the  Tutor  following  regular  instruction  in  tour  course  units  (one-dimensional 
motion,  two-dimensional  motion,  one-dimensional  dynamics,  ami  dynamics).  In  most 
instances,  students  worked  on  Tutor  problems  initially  in  pairs  and  later  independently. 
The  rationale  for  this  procedure  was  both  practical,  to  increase  practice  opportunities  on  the 
available  computers;  and  pedagogical,  to  give  students  the  opportunity  to  help  each  other 
during  the  beginning  stages  of  problem-solving. 

The 

"conventional  problems"  control  group  (n  =  13)  was  established  in  the  first  period  class, 
which  was  the  smallest  in  size  and  the  highest  in  ability.  This  group  received  conventional 
textbook  and  teacher-made  problems  on  the  selected  units. 

The  second- 
period  <'lass  consisted  of  25  students.  It  received  print  versions  of  the  same  Physics  Tutor 
problems  administered  to  the  PT  group  and,  like  the  PT  group,  worked  on  the  problems 


initially  in  pairs  and  later  independently.  However,  these  students  were  not  afforded  the 
advantages  of  immediate  feedback,  diagnosis,  and  adaptive  problem  selection  available  to 
the  PT  group.  This  group  will  be  referred  to  as  the  "no-feedback"  group.  Both  this  group 
and  the  control  group  had  limited  computer  activity  through  their  work  with  a  simulation 
program  early  in  the  year.  Neither  group  had  any  exposure  to  the  Tutor. 

Demographic 

and  pre-course  achievement  data  were  available  for  describing  and  comparing  the  three 
classes  on  (a)  the  mathematics  subtest  of  the  Stanford  Achievement  Test  taken  the  previous 
spring,  (b)  a  knowledge  pretest  administered  in  the  physics  class  at  the  beginning  of  the 
year,  and  (c)  the  highest-level  mathematics  course  completed.  Inferential  statistical  analysis 
(one-way  analysis  of  variance)  was  used  for  comparing  classes  on  these  pre-course  variables. 
Although  the  PT  and  no-feedback  class  scored  directionally  lower  than  the  control  class  on 
the  two  tests,  no  significant  differences  were  indicated. 
Instrumentation 

Two  times 

during  the  year  (September  and  November),  a  reaction  survey  was  administered  to  students 
in  class.  Part  1  of  the  survey,  which  was  administered  to  all  three  classes,  consisted  of  six 
questions  dealing  with  feelings  about  physics  and  problem  solving.  Examples  are  "I  like 
learning  physics"  and  "I  feel  confident  about  my  ability  to  solve  physics  problems." 
Responses  were  made  on  a  five-point  Likert  scale,  ranging  from  "strongly  disagree"  to 
"strongly  agree." 

Two  additional 

sections  of  the  survey  were  administered  to  PT  students  only.  Part  II  contained  six  ratings 
items  dealing  specifically  with  experiences  in  using  the  Tutor;  e.g.,  "The  procedures  for  using 
the  Tutor  were  easy  to  follow"  and  "I  prefer  the  Tutor  problems  to  doing  textbook  problems." 
Part  III  consisted  of  two  open-ended  questions  asking  what  was  liked  best  and  least  abort 
using  the  Tutor  for  the  current  lesson. 

An  overall 

survey  was  administered  to  the  PT  class  in  February,  several  weeks  after  they  had 
completed  their  last  Tutor  exercises.  This  survey  consisted  of  24  ratings  items  dealing  with 
specific  features  of  the  Tutor  (e.g.,  understandability,  operation  of  the  mouse  and  calculator, 
problem  selection,  graphics). 

In  addition  to 

the  survey,  approximately  five  randomly  selected  students  from  each  of  the  classes  were 
interviewed  in  September  and  November,  For  the  two  control  groups,  the  interview 
consisted  of  four  questions  concerning  attitudes  toward  phynicR  as  a  subject,  solving  physics 
problems,  and  their  physics  course.  Students  in  the  lyr  group  wore  asked  those  questions 
along  with  five  additional  questions  concerning  experiences  in  using  the  Tutor:  activities, 
likes  and  dislikes,  impact  on  problem  solving,  nnd  suggestions  for  improvement. 

During  the  fail 

semester,  the  teacher  kept  a  journal  in  which  she  recorded  her  impressions  and  activities  in 
using  the  Tutor  in  the  sixth-period  class,  Information  about  the  curriculum,  instructional 
objectives,  and  student  characteristics  was  also  given. 

Students' 

semester  test  results,  as  scored  by  the  teacher,  were  recorded  for  analysis  purposes.  The 
specific  tests  were  1st  six-weeks  test,  2nd  nix-weeks  lest,  Wni  six-weeks  test,  semester 
exam,  and  final  average.  The  scores  analyzed  were  total  scores  (0  100%)  without  any 
breakdowns  for  subsections  or  specific  problems.  Also,  to  explore  whether  students  in  the 
three  classes  differed  in  their  approaches  to  phynicH  problem  solving,  a  scheme  was  devised 
by  the  class  teacher  (Beverly  U»e)  and  t  he  Tutor  developer  (IV  I^oft  in)  for  evaluating  how 


individual  problems  were  set  up  and  worked.  The  system  involved  identifying  specific  steps 
or  features  of  a  correct,  systematic  problem-solving  procedure  for  the  particular  problem 
types  administered  (e,g.,  statement  of  correct  formula,  identification  of  variables,  etc.). 
Disc  ussion  and  informal  tryout  of  an  initial  model  led  to  the  development  of  the  final  scheme 
consisting  of  25  features  (e.g.,  initial  position,  velocity,  final  position,  etc.). 
Two  physics  teachers  from  local  high  schools,  who  were  trained  to  use  the  scheme, 
demonstrated  a  very  high  rate  of  agreement  (Overall  Af  =  94%)  in  their  scoring. 


judges  evaluated  28  problems  that  were  selected  from  different  tests  on  the  basis  of  having 
vs  tried  degrees  of  structural  similarity  with  actual  Tutor  problems.  Specifically,  some  were 
exact  repetitions  of  Tutor  problems,  and  some  were  similar  to  Tutor  problems  but  contained 
different  contexts.  Analyses  examined  results  on  individual  problems  and  on  clusters  of 
problems  representing  similar  problem  types  (e.^.,  one-dimunsional  kinematics,  two- 
dimensional  dynamics,  etc.). 

Results 

Student  Surveys 


student  surveys  showed  generally  positive  and  consistent  reactions  toward  physics  as  a 
subject  and  the  physics  class.  No  significant  differences  between  classes  were  found  on  any 
of  the  items.  The  strongest  trend  for  class  variations  was  indicated  on  Item  2-Round  2, 
"Confident  in  ability."  Total  agreement  ("agree"  +  "strongly  agree")  on  that  item  was  highest 
for  the  PT  group  (94%)  compared  to  the  no-feedback  (84%)  and  control  (64%)  group.  Also 
noteworthy  was  that  the  PT  and  no-feedback  classes  became  somewhat  more  negative 
across  rounds  about  working  with  a  partner  (from  6%  and  4%  disagreement,  respectively,  to 
20%  and  11%  disagreement).  In  general,  the  PT  group  was  somewhat  more  negative 
toward  physics  and  problem -solving  during  Round  1  than  were  the  two  control  classes,  but 
was  comparable  to  the  controls  by  Round  2. 


the  PT  group  to  the  Tutor  were  highly  positive  on  both  rounds.  In  fact,  the  highest 
percentage  of  students  expressing  disagreement  on  any  item  was  only  18%  which  occurred 
on  Round  1  in  reacting  to  "able  to  work  problems"  and  "prefer  Tutor  to  homework."  In 
Round  2  the  highest  level  of  disagreement  was  13%,  on  "procedures  were  easy  to  follow."  In 
reacting  to  the  statement,  "Overall,  I  liked  using  the  Tutor"  92%  of  the  students  agreed 
(only  1  disagreed).  Specific  features  that  received  the  highest  percentage  of  agreement 
responses  were  "helped  to  accurately  read  problem"  (92%),  "helped  me  learn  to  set  up 
problems"  (85%),  "liked  Yabha  Dabha  Doo  sound"  (85%),  "initial  instructions  were 
sufficient"  (84%),  and  "makes  physics  fun"  (83%). 

On  the  other  hand,  only  53%  percent  found  the  calculator  "easy  to  use,"  48%  agreed  that 
"feedback  helps  to  identify  mistakes,  "16%  thought  the  "help  button  offers  useful 
assistance,"  and  62%  agreed  that  the  "screen  design  tends  to  clutter  as  I  work."  These  less 
positive  reactions  seem  at  least  partly  attributable  to  students'  lack  of  familiarity  with  the 
sof  tware.  In  any  ease,  they  indicate  relatively  mild  degrees  of  dissatisfaction  and  are  clearly 
outweighed  by  the  clearly  positive  reactions  on  nearly  all  items. 
Student  Interview 


interview  responses  supported  the  survey  findings  by  showing  generally  positive  attitudes 
toward  the  physics  course  and  considerable  liking  of  the  Tutor.  When  asked  what  they 
liked  best  about  the  Tutor,  students  most  frequently  mentioned  the  immediate  feedback. 
Other  responses  given  by  more  than  one  student  were  "easier  to  work  with  formulas/'  "gives 


The  same  two 


Results  on  the 


Reactions  by 


Student 


step-by  step  instruction,  and  th*  "picture  helps  to  understand  the  problem."  Least  liked 
features  identified  by  more  than  one  student  were  "too  slow,"  "too  many  windows,  "and 
"cnlcultitor  is  hiird  to  use." 

The  last 

question  wuh  whether  the  Tutor  helped  them  to  become  better  problem  solvers.  Nearly  all 
studentH  strongly  agreed.  Among  the  reasons  given  were  demonstrations  of  the  problem, 
review,  and  immediate  feedback.  There  was  a  clear  difference  between  higher-  and  lower- 
achievers  in  the  reactions  expressed.  Students  who  felt  comfortable  about  their  problem- 
solving  abilities  felt  that  the  Tutor  was  useful  for  initial  learning  and  practice,  but  slowed 
them  down  once  they  gained  proficiency.  Specifically,  they  didn't  feel  that  they  needed  to 
work  through  all  the  steps  that  the  Tutor  required.  They  understood  those  steps  well 
enough  to  skip  them.  As  "expert"  problem  solvers,  they  were  at  the  stage  where  they  relied 
on  heuristics  (general  strategies)  and  did  not  need  step-by-step  procedures. 

Contrasting 

views  were  expressed  by  four  students  who  described  themselves  as  "struggling"  due  to 
weaknesses  in  mathematics  background.  They  felt  that  the  Tutor  was  extremely  helpful  at 
all  stages  of  the  learning  process.  The  main  advantage  was  guiding  them  through 
systematic  problem-solving  procedures,  while  not  allowing  them  to  proceed  down  an  incorrect 
pathway.  Because  of  these  features,  they  felt  that  their  approaches  to  problem-solving  were 
changing.  Specifically,  they  were  more  confident,  more  reflective  and  r  'stematic,  and  less 
concerned  about  mathematics  than  about  procedures.  All  felt  that  they  were  able  to 
transfer  these  new  approaches  to  problem-solving  to  textbook  and  examination  problems. 
Teacher  Jow    il  and  Interview  Reactions 

The  miiin 

advantage  of  the  Tutor,  in  the  teacher's  view,  was  helping  students  to  become  more 
systematic  problem  solvers  by  writing  down  correct  formulas  and  input  variables  before 
attempting  solutions.  For  example,  on  October  21,  the  teacher  wrote: 

One  student  had  not  identified  the  initial  horizontal  velocity  (mime  mugtiitiule).  The 
program  would  not  allow  him  to  begin  working  with  a  formula  until  he  luid  identified 
that  piece  of  information,  since  it  would  be  needed  later.., Tho  (error)  messages  he 
had  received  oil  referred  to  the  concept  of  velocity. ..Once  he  had  identified  that  value 
[Vox]*  he  could  use  the  formula.  This  la  exactly  what  the  program  should  bo 
doing!!!  The  student  received  the  correct  messages  and  hud  to  THINK  [bold 
type  and  capitalization,  hers]. 
Later  (November  8),  the  teacher  discussed  how  PT  students  began  to  accept  (though 
-"luctantly  at  first)  the  need  to  complete  each  step,  no  matter  how  small  or  obvious,  in  the 
problem-solving  process.  Students  in  the  control  classes  were  seemingly  less  receptive  to 
and  less  practiced  at  this  orientation. 

Not  all  aspects 

of  the  Tutor  experience  were  positive,  however.  Sometimes  (November  3)  the  Tutor  problems 
were  too  few  in  number  or  did  not  operate  correctly.  Other  limitations  were  excessive  time 
needed  for  equipment  set-ups,  equipment  failures,  and  minor  problems  that  students 
experienced  with  the  interface  (getting  lost  in  "too  many  windows").  The  teacher's  feeling 
was  that  these  difficulties  were  infrequent  and  minor  in  importance.  She  expected  their 
impact  to  increasingly  diminish  as  students  and  teachers  became  more  familiar  with  using 
the  Tutor  and  the  few  remaining  "bugs"  in  the  software  were  corrected. 

In  the 

int  erview,  the  teacher  expressed  the  overall  feeling  that  this  year's  uses  of  the  Tutor  were 
much  better  than  last  year's.  Her  hope  was  that  students  would  be  able  to  transfer  what 
they  had  learned  about  problem-solving  to  paper-and-pencil  problems.  Although  the  steps 
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Mint  the  TuUit  riM|in iimI  liMtk  extra  time,  that  sacrifice  was  clearly  worthwhile  if  student h 
1)€»c*ji ii)t*  luurr  nvhf  eiiuilir  |inililt*tn  solvers.  In  her  opinion,  the  high  achiever  who  wan 
slit'cessl'lil  with  both  procedures  and  answers  probably  did  not  need  the  Tutor  nearly  iih 
much  us  the  lower  achiever. 

Based  on  her 

early  observations  nf  1 1 *mI  results  (in  November),  she  felt  that  some  students  did  not  tranHfer 
the  problem-striving  procedure  outside  the  problem  domain.  With  additional  practices 
however,  they  seemed  to  perform  better,  so  there  was  hope  that  transfer  would  increase 
with  experience. 

The  teacher 

folt  thut  students  looked  forward  to  the  days  when  the  computer  was  used  and  to  their 
turns  on  it.  Without  cooperative  learning,  it  would  probably  have  been  more  difficult  for 
them  to  learn  the  problem-solving  skills.  They  conversed  extensively  before  making 
responses,  especially  on  new  or  complex  problems. 

In 

characterizing  the  overall  experience,  the  teacher  indicated  that  "there  have  been  no 
disappointments."  Set-up  work  and  equipment  problems  were  more  of  an  investment  at  the 
beginning  of  the  year  and  with  new  topics.  While  she  was  very  pleased  with  having  access 
to  the  Tutor,  she  did  not  feel  that  she  substantively  changed  her  teaching  style  in  employing 
it.  Nor  did  she  believe  that  the  primary  value  or  impact  of  the  Tutor  lied  in  engendering 
such  changes.  That  is,  whether  or  not  the  Tutor  were  available,  she  would  have  maintained 
her  primary  methods  of  lecture,  demonstration,  cooperative  practice  and  independent 
practice.  What  the  Tutor  did  provide,  in  her  view,  was  a  highly  valuable  tool  for  facilitating 
applications  of  these  strategies  through  its  functions  for  demonstration,  practice,  feedback, 
and  record-keeping.  While  qualitative  changes  in  teaching  methods  may  not  occur,  the 
teacher  believed  that  they  did  take  place  in  student  learning,  namely,  the  adoption  of  more 
reflective  and  systematic  problem -solving  approaches. 
Achicvummit 

Class 

performances  were  computed  on  the  following  achievement  tests:  1st  six-weeks,  2nd  six- 
weeks,  Urd  six-weeks,  semester  exam,  and  final  average.  To  determine  whether  group 
differences  wore  statistically  significant,  a  one-way  ANOVA  was  conducted  on  each  measure. 
None  of  the  achievement  comparisons  was  significant,  thereby,  suggesting  that  the  three 
classes  performed  comparably  on  the  five  overall  achievement  measures.  Nor  were  there  any 
significant  class  differences  when  test  scores  were  adjusted  for  pre-courso  aptitude  variables. 

Problem-Solving 

Problem- 
solving  approaches  by  students  in  the  three  classes  were  analyzed  in  several  ways.  The 
most  narrow  and  molecular  data  category  consisted  of  mean  scores  on  individual  elements 
across  all  problems.  A  second,  broader  category  examined  problem  means  by  tabulating  the 
number  of  elements  represented  in  the  given  problem  solution.  The  third  and  broadest 
category  combined  similar  problems  representing  a  certain  type  of  application  (e.g.,  1D- 
kinematics,  2D-  kinematics,  etc.)  to  yield  an  overall  problem  type  score. 

For  problem 

scores.,  results  showed  an  uneven  pattern,  with  significant  class  differences  obtained  on  only 
5  out  of  the  28  problems  and  on  the  overall  score  (points  summed  across  all  problems).  On 
the  five  significant  problems,  the  PT  group's  adjusted  mean  was  directionally  highest  on 
one,  second  highest  on  one,  and  lowest  on  three.  The  no-feedback  group  was  highest  on 
three,  second  highest  on  one,  and  lowest  on  one.  The  control  group  was  highest  on  one, 
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second  highest  on  three,  and  lowest  on  one.  On  total  score,  the  order  of  groups  was  no- 
fecdbuck  (A/=  407),  PT  (Af  =  394),  and  control  (M  =  387). 

For  problem- 
type  scores,  only  one  out  of  the  five  comparisons  was  statistically  significant.  On  the  2D 
kinematics  problems,  the  PT  group  was  lowest  and  the  no-feedback  group  was  highest. 
However,  in  an  effect  that  approached  significance,  the  PT  group  was  highest  on  2D- 
dynamics-inclined  plane  problems,  whereas  the  control  group  was  lowest.  The  direction  of 
the  means  on  the  various  problem  types  indicated  a  trend  for  the  PT  group  to  score 
relatively  higher  compared  to  the  other  groups  as  problem  complexity  (and  course  experience) 
increased.  Specifically,  PT  scores  were  lowest  of  the  three  groups  on  1-D  kinematics 
problems,  but  highest  on  the  more  difficult  2-D  dynamics  problems. 

For  element 

scores,  significant  effects  were  obtained  on  18  out  of  187  elements  associated  with  1-D 
kinematics  problems  (the  easiest  level).  The  direction  of  the  means  showed  the  PT  group  to 
typically  be  lowest  of  the  three  groups  (in  72%  of  the  cases),  and  the  no-feedback  group  to  be 
highest  (in  94%).  Results  for  2-D  kinematics  problems  showed  significant  differences  on  20 
out  of  56  elements.  As  with  1-D  kinematics  problems,  the  PT  group  typically  scored  lowest 
(in  95%  of  the  cases)  and  the  no-feedback  group  highest  (in  65%).  On  1-D  kinematics  with  1- 
D  dynamics  problems,  9  out  of  56  elements  showed  significant  group  effects.  Here,  the 
results  were  less  consistent,  with  the  PT  group  scoring  highest  on  2  elements  (22%),  second 
highest  on  4  (44%),  and  lowest  on  3  (33%).  Finally,  significant  group  differences  were 
obtained  on  17  out  of  the  191  elements  on  the  2-D  dynamics  problems:  the  PT  group  scored 
the  highest  of  the  three  groups  on  15  elements  (88%).  The  no-feedback  group  was  second  in 
nearly  all  cases. 

Conclusions 

Conclusions 

based  on  the  results  for  student  attitudes,  teacher  experiences,  and  student  problem-solving 
outcomes  are  described  below. 
Student  Attitudes 

Judging  from 

our  five  independent  visits  of  PT  classes  during  the  past  two  years,  and  student  survey  and 
interview  responses,  there  is  no  question  that  students  felt  very  positively  about  using  the 
Xutflt'   They  greatly  looked  forward  to  "Tutor  periods"  as  opportunities  to  break  from  the 
routine  of  working  problems  by  hand  without  the  benefit  of  error- sensitive  feedback. 
Working  with  a  computer  appeared  to  be  as  motivating  for  these  advanced  high  school 
students  as  for  the  elementary  school  children  we've  observed  in  numerous  contexts.  There 
is  also  no  question  that  the  cooperative  learning  format  used  by  the  teacher  enhanced  the 
experience  for  most  students,  especially  lower-achievers  who,  if  forced  to  work  alone,  would 
have  struggled  with  identifying  correct  problem  solving  steps.  The  latter  impression  is 
important  in  considering  instructional  uses  of  the  Tutor,  given  that  the  Tutor  primarily 
"reacts"  to  student  responses  hut  does  not  directly  "teach"  the  physics  concepts  needed  to 
understand  the  problem.  Cooperative  learning  allows  students  to  discuss  (and  teach  each 
other)  the  problem-solving  strategies  and  principles,  and  then  seethe  results  of  their 
decisions. 

Negative  reactions  to  the  Tutor  were  infrequently  expressed.  In  fact,  not  one 
PT  student  indicated  that  he/she  preferred  conventional  methods  over  the  PT  problem- 
solving  activities.  The  dissatisfactions  that  were  conveyed  tended  to  concern  minor  aspects 
of  the  user  interface  that  made  the  Tutor  le9s  convenient  to  use  than  paper-and-pencil 
modes.  The  most  noteworthy  of  these  concerned  the  ease  of  using  the  calculator  and  the 
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llflp  function,  the  slowness  of  I  hi1  system,  mui  the  cluttering  of  the  display  windows.  These 
fiic tors  seemed  to  become  less  important  as  students  acquired  more  experience  in  using  the 
Tutor.  In  the  November  interviews,  for  example,  several  students  characterized  the 
interface  pr'K'cdureH  as  difficult  (or  new)  (it  first,  hut  "second  nature"  now. 

Our  overall  conclusion  is  that  no  major  revisions  in  the  user  interface  are 
indicated  by  the  evaluation  results.  The  program  developers  should  review  the  student 
reactions,  in  relation  to  considerations  of  the  time  and  cost  requirements  of  associated 
programming  changes,  ami  decide  which,  if  any,  revisions  would  be  desirable. 

An  additional  conclusion  supported  by  the  student  reactions  is  that  the 
Tutor  offers  greater  benefits  to  lower-achieving  than  to  higher-achieving  students.  This 
interpretation  is  hardly  surprising,  given  that  high  achievers  are,  by  definition,  relatively 
successful  learners  who  have  performed  well  using  traditional  learning  methods  and 
resources.  The  literature  on  problem  solving  (e.g.,  Chi,  Feltovich,  &  Glaser,  1982)  suggests 
that  expert  problem  solvers  tend  to  use  a  "top-down"  approach,  which  emphasizes 
understanding  of  problem  patterns  and  contexts,  as  opposed  to  a  "bottom-up"  approach, 
which  emphasizes  the  piece-by- piece  building  of  the  problem  solution.  From  this 
perspective,  it  is  reasonable  that  the  highest-achieving  students  sometimes  felt  constrained 
by  the  step-by-step  procedures  that  the  Tutor  enforced.  They  felt  that  they  were  able  to  skip 
many  of  these  steps  and  consequently,  solve  the  problem  more  efficiently  using  paper  and 
pencil.  All  students  agreed,  however,  that  beginning  a  unit  with  a  few  Tutor  problems 
served  a  valuable  function,  by  exposing  them  to  a  complete,  systematic  procedure. 

It  is  important  to  remember  that  the  present  subjects  were  all  physics 
honors  students  attending  a  middle-  to  upper-middle-class  suburban  school.  Thus,  the 
proportion  of  high-achievers,  who  experience  success  working  independently,  was  likely  to  be 
substantially  greater  for  this  sample  than  would  occur  in  "regular"  physics  classes  at  other 
schools. 

Teacher  Attitudes 

As  conveyed  by  the  teacher  journal  and  interview  results,  the  teacher  was 
very  satisfied  with  the  Tutor  software.  She  shared  the  students'  perception  that  the  Tutor 
problems  were  especially  valuable  at  the  beginning  of  a  new  unit  and  for  students  who  were 
experiencing  difficulties.  During  the  year,  she  encountered  few  difficulties  with  the  operation 
of  the  software  or  hardware. 

Because  only  one  teacher  participated  in  the  evaluation,  it  is  risky  to  make 
general  conclusions  about  how  other  teachers  would  utilize  or  react  to  the  Tutor.  Some 
teachers,  for  example,  might  prefer  individual  problem  solving  over  cooperative  activities; 
some  might  use  the  class  and  individual  feedback  reports  to  a  greater  or  lesser  extent, 
(liven  the  cane  of  using  the  Tutor  and  the  few  problems  that  were  encountered  in  the 
present  application,  our  strong  impression  is  that  most  teachers  would  react  quite  positively 
to  thi<  Tutor  and  welcome  its  availability  for  their  classes. 

Unique  to  the  present  application,  however,  was  the  teacher's  long-term 
involvement  with  the  development  of  the  Tutor,  interest  in  the  project,  and  access  to  the 
developers  who  worked  locally  at  NASA.  New  users,  not  associated  with  the  project,  would 
he  less  knowledgeable  about  and  adept  at  using  the  program.  Support  material,  such  as  a 
"user's  manual/'  ih  needed  to  provide  guidance  on  both  operating  procedures  and 
instructional  methods. 
AriiiuvunuU  Unttunuti 

Hosults  on  the  various  problem-solving  tests  did  not  indicate  significant 
advantages  for  the  PT  group.  The  latter  class  began  the  year  as  the  lowest  of  the  three 
classes  in  physics  aptitude  and  mathematics  experience,  and  completed  the  year  in  the 
same  position  as  measured  by  examination  scores.  This  outcome  should  not  be  interpreted 
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negatively,  but  as  inconclusive  in  view  of  the  constraints  of  the  present  program  context, 

First,  the  Tutor  was  utilized  for  an  extremely  small  proportion  of  the 
instructional  time  during  the  year.  It  thus  seems  unreasonable  to  expect  it  to  impact 
performances  that  are  likely  to  be  influenced  by  so  many  confounding  variables  (e.g.,  student 
preparation  for  tests,  learning  from  conventional  teaching  methods  and  other  resources 
homework,  practice  problems,  cooperative  learning,  learner  characteristics,  class  size,  etc.), 

Second,  subjects  consisted  exclusively  of  honors  students,  thus  representing  a 
relatively  homogeneous  sample  of  students  who  generally  perform  well  academically. 
Chances  of  treatment  differences  occurring  with  this  select  population  would  be  greatly 
reduced  relative  to  using  low-  or  average-achievers. 

Third,  chance  factors  resulted  in  an  unevea  distribution  of  class  sizes  and 
abilities  in  the  three  comparison  groups.  The  control  (first-period)  class,  for  example,  was 
the  smallest  and  highest  in  ability  of  the  three. 

Fourth,  the  Tutor  was  primarily  designed  to  influence  students'  approaches 
to  problem-solving,  by  reinforcing  systematic  solution  steps  and  recording  of  step-by-step 
mathematical  work.  Such  influences  may  increase  "general"  or  long-term  problem-solving 
ability,  while  having  little  impact  on  immediate  test  scores  based  on  a  restricted,  known 
group  of  problems.  Seemingly,  short-cut  or  idiosyncratic  Holution  strategies  can  be  used  on 
the  latter  problems  with  much  greater  success  than  would  be  case  for  transfer  problems 
from  many  different  domains. 

Based  on  the  above  considerations,  we  ronsider  tin1  evaluation  of  Tutor 
effects  on  course  achievement  to  be  inconclusive,  Further  evaluation  is  recommended,  using 
matched,  regular  physics  classes  over  a  longer  treatment  period, 
Problem-Solving  Strategy 

Results  from  the  analyses  of  problem-solving  strategies  were  highly 
suggestive  regarding  the  potential  of  the  Tutor  to  influence  how  students  solve  problems. 
These  results  were  also  consistent  with  both  student  and  teacher  perceptions  of  Tutor 
effects. 

The  pattern  of  behavior  indicated  by  the  problem -solving  measure  revealed 
little  change  by  the  PT  group  early  in  the  semester,  in  solving  the  beginning  one-  and  two- 
dimensional  kinematics  problems.  Consistent  with  their  low  standing  on  tho  pre-course 
variables,  the  PT  group  had  the  lowest  scores  of  the  three  groups  in  representing  relevant 
elements  in  their  problem  solutions.  As  the  semester  progressed,  and  experiences  with  the 
Tutor  increased,  this  pattern  reversed  to  reflect  increasing  use  by  the  PT  group  of  the 
problem  elements.  On  the  last  and  most  advanced  unit,  two-dimensional  dynamics,  the  PT 
students  had  the  highest  problem-solving  strategy  scores  of  the  three  groups. 

These  results  suggest  that  the  Tutor  is  effective  at  developing  more 
systematic  problem-solving  strategies.  Such  influences,  however,  take  time  to  incubate  and 
may  therefore  may  not  emerge  until  later  units  of  study.  It  is  also  noteworthy  that  the  no- 
feedback  control  group  scored  relatively  high  on  the  strategy  measure.  The  implication  is 
that  the  use  of  print  versions  of  the  Tutor  problems,  coupled  with  teacher  feedback,  was  also 
effective  (perhaps  more  in  the  beginning,  since  such  feedback  may  have  implicitly  or  explicitly 
conveyed  the  teacher's  expectancy  to  see  complete  representations  used  on  subsequent 
problems.)  However,  the  disadvantage  relative  to  the  Tutor,  in  the  long-run  is  the  difficulty 
for  teachers  of  adapting  feedback  to  individuals  as  problem  complexity  and  the  variability  of 
student  needs  increase  over  time. 
Recommendations 

Despite  the  failure  to  show  differences  in  problem -solving  (test)  scores,  the 
evaluation  findings,  overall,  were  positive  regarding  the  Tutor's  utility  for  instruction  and 
effectiveness  for  student  learning.  The  following  recommendations  are  made  for  further 
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development  of  and  research  with  the  Tutor. 

1.  The  developers  should  review  the  student  and  teacher  evaluations 
regarding  the  user  interface,  and  make  whatever  minor  improvements  they  consider 
desirable. 

2.  Cooperative  learning  should  continue  to  be  recommended  as  a  desirable 
instructional  strategy  for  using  the  Tutor. 

3.  A  user's  manual  and  teacher's  manual  (or  a  combination)  should  be 
developed  to  provide  clear  information  about  the  program  (instructions,  set-ups,  operating 
procedures,  etc.)  and  guidelines  for  instructional  uses. 

4.  Additional  evaluation  should  be  conducted  at  different  schools,  using 
regulur  (not  honors)  classes  that  are  matched  in  ability  and  size. 
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Introduction 

Computer  based  learning  systems,  especially  those  which  have  as  design  goals  a 
high  degree  of  learner  control,  generally  require  a  correspondingly  high  degree  of  motivation 
in  the  learners  involved.  As  an  important  and  long  recognized  element  in  learner 
motivation,  curiosity  thus  becomes  a  significant  concern  for  designers  of  highly  interactive 
computer-based  learning  environments  (Kinzie,  1990).  However,  the  role  that  curiosity 
plays  in  the  learning  process  seems  so  integral  that  its  importance  is  often  assumed.  In 
recent  research  literature  (with  few  exceptions)  the  term  is  either  applied  loosely  or  is 
subsumed  into  the  general  body  of  work  on  learner  motivation,  especially  as  it  pertains  to 
intrinsic  motivation.  With  the  exception  of  those  investigators  engaged  in  ongoing  research 
with  very  young  children,  the  work  of  Berlyne  and  Maw  &  Maw  in  the  50's  and  60*s  remain 
the  standard. 

Recently  Arnone  and  Grabowski  (1992,  1993)  have  begun  to  investigate  the  role  of 
curiosity  in  computer-based  interactive  video  lessons.  In  addition,  other  investigators  have 
continued  to  pursue  more  broadly  empirical  and  theoretical  work  in  curiosity,  building  upon 
Berlyne's  seminal  work  in  the  field  (Wohlwill,  1987).  Of  particular  note  to  this  paper  is  the 
work  on  "flow"  experiences  by  Csikszentmihalyi  (1975,  1990a;  Whalen  &  Csikszentmihalyi, 
1991).  His  work  provides  insight  into  the  kinds  of  engagement  experienced  by  learners 
whose  curiosity  has  led  them  deeply  into  a  lesson.  In  the  light  of  these  investigations  and 
theorizing,  this  research  paper  seeks  to  investigate  the  phenomenon  of  curiosity  in  learners 
involved  in  a  highly  interactive,  computer-based  multimedia  lesson. 

This  paper  will  examine  the  relationships  between  curiosity,  intrinsic  motivation, 
and  Csikszentmihalyi's  "flow"  state  of  absorbed  participation.  Specifically,  this  study  of  the 
literature  will  investigate  the  possible  roles  curiosity  plays  in  the  triggering  and 
maintenance  of  a  flow  state  amongst  learners  engaged  in  an  interactive  lesson. 
Establishing  a  role  for  curiosity  in  flow  states  will,  it  is  hoped,  provide  further  insight  into 
how  this  potent  motivational  tool  can  be  better  utilized  by  designers  of  instruction. 

Background 

In  his  often  cited  work  Conflict.  Arousal,  and  Curiosity  (1960),  Berlyne  formalized 
curiosity  as  "exploratory  behavior"  and  posed  the  existence  of  a  number  of  kinds.  Generally, 
he  grouped  them  into  two  broad  types.  Included  in  the  first  type  were  those  that  were  overt 
responses  to  external  stimuli  (or  the  lack  of  them)  which  he  called  "perceptual  responses." 
The  second  type  he  termed  "epistemic  responses"  and  he  typified  them  as  being  an 
internalized  search  for  some  of  a  number  of  kinds  of  knowledge  by  the  person.  He  admitted 
these  two  types  "may  often  coincide,"  clearly  indicating  the  distinctions  were  for  purposes  of 
discussion  and  analysis  alone  (p.  80). 

Berlyne  (1960)  saw  organisms  as  continually  seeking  to  maintain  an  optimum  state 
of  arousal  and  that  exploratory  behavior  s  purpose  was  to  either  boost  or  reduce  an 
unacceptable  level  of  arousal.  In  the  case  of  epistemic  behaviors,  arousal  was  caused  by  a 
•'conceptual  conflict,"  a  term  Berlyne  synonymizes  with  Piaget's  "disequilibrium"  (p.  220)  and 
Kestinger's  "cognitive  dissonance"  theory  (p.  283).  He  saw  a  search  for  knowledge  as  a 
search  for  the  assuagement  of  a  too  high  or  too  low  state  of  arousal  caused  by  some  sort  of 
conceptual  conflict  engendered  by  the  person's  mental  or  physical  activities  of  the  moment. 
"Epistemic  curiosity"  is  the  term  Berlyne  applies  to  the  result  of  conceptual  conflict  occurring, 
and  a  search  for  knowledge  is  the  behavior  that  results.  The  internalized  nature  of  this 
stimulus-response-reward  process  places  Berlyne's  curiosity  theorizing  in  the  middle  of 
almost  any  discussion  of  intrinsic  motivation  (Day,  Berlyne,  &  Hunt,  1971;  Deci,  1975; 
Keller,  1987). 

The  work  of  Maw  and  Maw  (1964,  1965)  centered  on  the  detailed  measurement  of 
curiosity  levels  in  individuals  (children)  and  on  classifying  observable  behaviors  that 
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evidenced  epistemic  curiosity  at  work.  They  defined  curiosity  based  on  the  presence  of  the 
foil  ^wing  behaviors: 

When  children- 

1 .  react  positively  to  new,  strange,  incongruous,  or  mysterious  elements  in  their 
environment  by  moving  toward  them,  by  exploring  them,  or  by  manipulating 
them 

2.  exhibit  a  need  or  desire  to  know  more  about  themselves  and/or  their 
environment 

3.  scan  their  surroundings  seeking  new  experiences 

4.  persist  in  examining  and  exploring  stimuli  in  order  to  know  more  about  them 
-they  are  being  curious  (1964,  p.  31) 

The  labels  "high  curious-'1  and  "low  curious  child"  were  then  used  to  investigate  possible 
correlations  with  such  factors  as  socioeconomic  status,  parental  attitudes,  creativity,  etc.  In 
these  pursuits  they  developed  instruments  that  served  to  objectify  the  curiosity  level  of  a 
child.  Their  definition  has  remained  useful  and  is  often  cited  to  this  day  (e.g.,  Arnone  & 
Grabowski,  1992;  Day,  1982). 

Many  researchers  since  then  have  continued  to  explore  the  measurement* 
classification-dissection  of  exploratory  behavior.  Some  researchers  have  tended  to  study 
curiosity  (in  children  especially)  alongside  of  "play,"  "imagina  ioj  *  and  "creativity."  Other 
theoreticians  have  grappled  with  the  question  of  whether  it  is  a  "state"  (i.e.,  a  condition 
brought  on  by  some  stimulus  or  process)  or  a  "trait"  (i.e.,  a  universal  psychological  need  or 
drive).  In  his  last  work,  Curiosity  and  Learning  (left  incomplete  by  his  death  in  1976), 
Berlyne  (1978)  repeatedly  called  for  the  development  of  a  theory  of  motivation  for  explaining 
human  behavior  (as  noted  in  Keller,  1987).  That  he  did  so  in  a  work  ostensibly  about 
curiosity's  role  in  learning  reveals  the  inextricable  relationship  he  saw  between  curiosity  and 
motivational  issues  in  learning. 

Motivational  psychologists  mirror  the  divided  opinions  about  curiosity  outlined  in  the 
preceding  paragraph.  On  the  one  hand,  a  divergence  from  the  person's  opt,  nal  level  of 
arousal  results  in  (motivates)  behavior  (called  curiosity)  which  seeks  to  restore  their  arousal 
level  (Berlyne,  1971).  On  the  other  itiand,  curiosity  is  seen  as  a  result  of  "cognitive 
incongruence"  for  which  persons  "inherently"  require  corrective  action  in  order  to  feel 
competent  (Deci,  1975;  Deci  &  Porac,  1978).  The  question,  as  before  with  the  nature  of 
curiosity  itself,  remains:  Is  intrinsic  motivation  a  state,  or  a  trait,  of  human  existence? 
While  there  are  areas  of  common  ground  between  these  two,  theoretical  discussion  continues 
(Keller,  1987;  Voss  &  Keller,  1983).  In  the  meantime,  those  with  a  more  practical  bent 
have  begun  to  apply  some  of  this  theorizing  to  everyday  practice. 

Csikszentmihalyi  (1975,  1990a)  has  posed  the  existence  of  a  state  of  human 
experience  he  calls  "flow"  wherein  the  individual  feels  (a)  a  merging  of  action  and  awareness, 
(b)  the  centering  of  attention  on  a  limited  stimulus  field,  (c)  self-forgetfulness  or  loss  of  self 
consciousness,  (d)  a  sense  of  control  over  one's  actions  and  their  environment,  (e)  the 
coherence  and  noncontradiction  of  demands  for  action  and  the  provision  of  clear, 
unambiguous  feedback  from  one's  actions,  and  (f)  an  "autotelic"  nature,  i.e.,  the  experience 
requires  no  goals  or  rewards  to  itself,  (from  Keller,  1987,  p.  35).  The  internal  nature  of 
"flow"  clearly  places  it,  as  a  phenomena,  well  within  the  general  arena  of  intrinsic 
motivation  studies  (Csikszentmihalyi,  1978).  In  addition,  such  an  experience,  fully  realized, 
also  represents  a  fair  description  of  the  desired  state  of  a  learner  fully  engaged  in  computer- 
based  instruction,  at  least  according  to  many  interested  in  the  development  of  such 
instruction  (Keller  &  Suzuki,  1988;  Kinzie,  1990;  Lepper  &  Malone,  1987). 

Curiosity  and  Instruction:  Research 

As  has  already  been  noted,  there  has  not  been  much  applied  research  done  with 
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human  subjects  concerning  curiosity  and  learning  in  the  ages  beyond  the  preschool  years. 
Opinions  and  attendant  theorizing  about  it  are  more  common,  but  unless  the  topic  involves 
motivation,  curiosity  is  either  broadly  assumed  or  the  research  effort  itself  is  concerned 
simply  with  recognizing,  defining  or  measuring  it  (Boykin  &  Harackiewicz,  1981).  In  the 
case  of  Maw  and  Maw,  whose  work  pioneered  the  field  of  curiosity  measurement,  their 
prescriptive  conclusions  summarized  and  reported  later  (Maw,  1971)  remained  very  general: 
"If  a  school  wishes  to  increase  curiosity,  so  essential  to  learning,  it  must  turn  its  attention  to 
seeing  that  each  child  feels...  worthy...  of  some  importance...  secure...  accepted...  loved,"  et?. 
(p. 97)  [emphasis  added].  Harty  and  Beall  (1984),  answering  the  same  call  as  Maw  and 
Maw  for  useful  curiosity  measuring  instruments,  engaged  in  the  creation  of  the  "Children's 
Science  Curiosity  Scale."  The  test  they  developed  went  through  six  versions  before  settling 
on  a  30  question  instrument  using  a  Likert  type  agree/disagree  five  point  scale.  They 
concluded  that  the  instrument  appears  to  be  reliable  and  valid  but  cautioned  that  further 
testing  with  larger  populations  was  needed  before  it  could  become  a  standardized  tool  for 
educators  to  use  (they  used  less  than  200  students  overall). 

With  the  notable  exception  of  Berlyne  himself,  other  research  applying  his  theories  of 
curiosity  to  actual  learning  situations  (in  humans)  remains  sparse  or  flawed  (Koran,  Koran, 
Foster,  &  Fire,  1989).  An  exception  is  the  work  of  Boykin  and  Harackiewicz  (1981).  They 
were  particularly  interested  in  discovering  how  subjects,  uncertain  of  their  abilities  to 
answer  a  question,  would  evidence  curiosity  as  a  response  and,  additionally,  how  their 
uncertainty  would  translate  into  later  recall  of  the  knowledge.  The  results  confirmed  the 
linear  relationship  between  uncertainty  and  curiosity  as  those  most  uncertain  expressed  the 
highest  curiosity.  Also,  the  problems  with  the  highest  likely  uncertainty  (i.e.,  the  most 
difficult)  elicited  the  highest  levels  of  curiosity.  Subjects  also  tended  to  remember  the  most 
difficult  words  in  the  recognition  test.  The  authors  concluded  that  the  supposed  relationship 
of  uncertainty  to  curiosity  and  of  uncertainty/curiosity  to  retention  was  confirmed.  The 
authors  felt  their  methodology  was  sound  and  that  their  instrument  could  be  used 
successfully  for  further  investigations  into  curiosity. 

Another  example  is  the  work  of  Inagaki  where  the  author  tested  a  number  of 
hypotheses  regarding  curiosity,  learner  effort,  and  performance  in  science  classes.  In  the 
first  study  (Inagaki  &  Hatano,  1977)  (using  fourth-graders)  the  treatment  group  (curiosity 
aroused)  showed  stronger  desire  for  confirmation  of  a  science  experiment  by  observation, 
better  generalization  of  the  knowledge  to  other  situations,  and  a  higher  correlation  between 
performance  and  curiosity. 

In  another  study  (using  kindergartners),  citing  the  theory  that  highly  curious  children 
will  interact  more  with  their  environments  and  thus  gain  increased  mastery  over  it,  Inagaki 
(1978)  posed  two  hypotheses:  (1)  The  higher  the  curiosity,  the  more  actively  the  subjects 
will  explore  an  experimental  environment  containing  uncertain  or  dissonant  elements;  and 
(2)  the  higher  the  curiosity,  the  more  measurable  information  the  subjects  will  acquire, 
regardless  of  verbal  ability.  The  results  indicated  a  degree  of  correlation  (but  not  a 
significant  one)  between  curiosity  and  amount  of  exploration.  High  curiosity  did  correlate 
significantly  with  performance,  even  with  verbal  ability  held  constant,  confirming  hypothesis 
No.  2.  The  author  concluded  that  curiosity  (as  a  trait)  played  a  significant  role  in  learning. 
He  goes  on  to  predict  that  highly  curious  students  will  be  more  apt  to  have  success  in 
"8tudent«centered  learning,"  i.e.  learner  controlled  environments. 

The  research  reviewed  here  (obviously)  represents  only  a  sample  of  the  empirical 
work  done  in  the  area.  However,  studies  tend  to  confirm  (a)  the  presence  of  "curiosity* 
through  observable,  quantifiable  behaviors,  (b)  that  its  presence  or  activation  results  in 
improved  learning/performance,  (c)  that  it  can  be  aroused  by  various  instructional 
techniques,  and  (d)  that  aroused  learners  will  persist  and  be  self-sufficient  in  their  pursuit  of 
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learning  goals  (Day,  1982).  Its  psychological  basis  and  attendant  cognitive  issues  remain 
open  to  debate,  however.  As  noted  before,  the  work  of  neonatal  and  infant  researchers  is 
the  cutting  edge  for  this  aspect  of  the  field.  Kor  the  rest,  including  those  involved  in  creating 
computer-based  interactive  instruction,  "curiosity"  in  all  its  theoretical  and  practical 
manifestations)  remains  an  elusive  target  to  hit,  albeit  an  important  one. 

Curiosi:>  and  Interactive  Instruction 
Curiosity's  role  as  a  motivator  in  computer-based  learning  has  been  an  object  of 
interest  for  some  time  (e.g.  Miller  &  Hess,  1972),  Currently,  curiosity  figures  prominently  in 
the  work  of  several  important  theorists  within  the  field  of  instructional  technology,  J.  M. 
Keller's  "ARCS  Model  of  Motivation"  (Keller  &  Suzuki,  1988)  and  Malone's  "Taxonomy  of 
Intrinsic  Motivations  for  Learning"  (1981;  Malone  &  Lepper,  1987)  each  find  a  place  for 
"curiosity"  in  their  respective  schemes  meant  to  explicate  the  role  of  motivation  in 
instructional  design. 

In  the  case  of  the  ARCS  model,  Keller  places  curiosity  in  the  "A"  for  "attention"  part 
of  his  scheme.  In  this  placement  he  emphasizes  the  ability  of  novelty,  uncertainty,  surprise, 
etc.  to  get  attention  (i.e.,  arouse  curiosity).  He  hopes  that  the  gaining  of  attention  will  be 
followed  by  engagement  of  the  learner  "in  an  inquiring  frame  of  mind."  This  deeper 
engagement  is  epistemic  curiosity,  Berlyne's  search  for  knowledge  (Keller  &  Suzuki,  1988,  p. 
410).  However,  a  close  examination  of  the  remainder  of  the  model,  Belevance£onfidence- 
Satisfaction,  reveals  those  elements  to  also  be  elements  in  the  curiosity  theories  posed  by 
Berlyne  and  others.  Novelty,  et  al,  are  insufficient  arousal  agents  in  and  of  themselves  to 
insure  the  kinds  of  epistemic  exploration  we  think  of  as  learning.  If  the  situation  does  not 
show  promise  of  being  useful,  moderately  non-threatening,  and  of  being  rewarding,  we  will 
not  be  led  to  explore  it  in  any  extended  way.  The  entire  ARCS  model  requires  "curiosity"  as 
a  motivator  for  learning,  at  least  as  Berlyne  (1971)  would  have  it. 

Malone  s  taxonomy  poses  a  more  comprehensive  set  of  motivators  than  Keller.  He, 
too,  places  curiosity  in  a  separate  category  from  the  other  catch  words  that  make  up  his 
theory.  In  his  discussion  of  it,  however,  he  states,  "In  a  sense,  curiosity  is  the  most  direct 
intrinsic  motivation  for  learning,"  and  draws  many  parallels  between  it  and  the  "challenge" 
element  of  the  taxonomy  (Malone  &  Lepper,  1987,  p.  235ff.).  In  his  first  discussion  of  his 
theory  he  states,  "In  fact,  challenge  could  be  explained  as  curiosity  about  one's  own  ability, 
or  curiosity  could  be  explained  as  a  challenge  to  one's  understanding,"  (Malone,  1981,  pp. 
362-363).  Clearly  this  relationship  has  much  in  common  with  the  idea  that  curiosity's 
purpose  is  to  assuage  a  perceptual  or  cognitive  conflict,  again  from  Berlyne  (1960). 

The  point  of  this  analyzing  is  to  highlight  the  centrality  of  curiosity  (at  least  as  a  full 
expression  of  Berlyne's  curiosity  theories  would  have  it)  to  those  two  attempts  at  delineating 
the  motivational  underpinnings  of  computer-based  instruction.  Keller,  especially,  is  limited 
in  his  appreciation  of  the  comprehensive  nature  o,% .  uriosity  in  relation  to  the  field  of 
motivation.  Perhaps  Berlyne,  the  father  of  curiosity  theorizing,  was  right  in  insisting,  "A 
theory  of  motivation  is  indispensable  if  quest  ions  of  performance  are  to  be  answered,"  (1978, 
p.  119), 

In  the  work  of  Arnone  and  (irabowski  (1992)  mentioned  earlier,  empirical 
researchers  have  begun  to  address  curiosity  and  its  role  in  computer-based  interactive 
instruction  directly.  In  their  study  they  focussed  on  how  variations  in  learner  control  affected 
their  subjects'  curiosity  and  achievement.  The  researchers'  concern  was  with  the  relative 
"amount"  of  curiosity  demonstrated,  similar  to  the  Maw  and  Maw  studies  in  the  60s. 

The  authors  created  a  lesson  on  art  museums  using  a  computer  controlled  videodisc 
with  a  touchscreen  monitor.  The  lesson  was  presented  in  three  versions:  (1)  program 
controlled;  (2)  learner  controlled;  and,  (3)  learner  controlled  with  advisement.  101  first  and 
second  graders  from  a  public  school  were  randomly  divided  into  four  groups,  one  for  each 
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treatment,  plus  a  base  (control)  group  that  received  no  instruction.  After  the  treatments  all 
groups  took  a  posttest  that  measured  achievement  and  three  different  curiosity  measures: 
content  exploration,  questioning,  and  persistence,  again  using  an  interactive  videodisc  setup. 

The  results  indicated  that  advisement  resulted  in  achievement  that  was 
significantly  better  than  without  advisement,  though  no  difference  was  found  between  these 
two  groups  and  the  program  control  group.  All  three  treatment  groups  significantly 
outperformed  the  base  (no  treatment)  group.  In  the  curiosity  measures,  there  were  no 
differences  in  the  "questioning"  measure,  Significant  differences  were  found  between  the 
base  group  and  the  others  in  "persistence"  with  the  base  group  persisting  longer.  A 
significant  difference  was  also  found  in  the  "content  exploration"  measure  between  the  base 
group  and  the  advisement  group  with  the  the  latter  showing  greater  curiosity. 

In  their  discussion  the  authors  took  interest  in  the  fact  that  the  learner  control 
group's  achievement  was  on  a  par  with  that  of  the  other  control  options,  possibly  due  to  the 
contents  inherent  interest.  They  also  found  the  non-instructed  base  group's  high  curiosity 
(as  measured  by  persistence)  indicative  of  their  lack  of  prior  exposure  to  the  content--  it  was 
all  new  to  them  and  they  took  their  time  (the  actual  measure  used)  examining  the  stimuli 
during  the  posttest.  On  the  other  hand,  the  advisement  group's  greater  curiosity  (as 
measured  by  content  exploration)  than  the  non-treated  base  group  represented  the  fruits  of 
being  instructed  to  "think"  about  the  content  during  the  instruction,  thus  making  them  more 
"intrigued"  about  museums  in  general. 

While  Arnone  and  Grabowski's  work  provides  insight  into  how  curious  (or  non- 
curious)  subjects  perform  in  interactive  instruction,  practitioners  are  more  interested  in  how 
curiosity  can  be  invoked  in  all  subjects  experiencing  computer-based  interactive  instruction. 
Arnone  and  Grabowski  (1993)  have  continued  their  work  in  this  area  specifically.  On  a 
broader  front,  Westrom  and  Shaban  (1992)  have  applied  Malone's  work  in  a  study 
comparing  the  motivational  levels  of  players  in  "instructional"  vs.  "noninstructional"  games. 
The  authors  found  all  of  Malone's  four  factors  (Challenge,  Curiosity,  Control,  Fantasy) 
played  roles  in  the  initial  and  continuing  engagement  of  the  players  /  students.  They 
concluded  that  a  better  understanding  of  motivational  effects  would  lead  to  more  effective 
instructional  tools  wherein  students  "might  find  that  learning  is,  once  again,  fun,"  (p.  444). 

While  many  factors  have  been  posed  as  contributors  to  learner  motivation  in  the 
field  and  several  theories  proposed  (i.e.,  the  Keller  and  Malone  theories  above),  curiosity 
remains  a  difficult  concept  for  most  authors  and  researchers  to  deal  with  concretely  in 
isolation.  Like  many  of  humankind's  affects,  it  is  easier  to  describe  what  it  does  than  what 
it  is.  Perhaps  we  need  to  forego  reductionist  approaches  and  take  up  those  that  are 
integrative  and  holistic  in  their  place.  An  example  of  such  an  approach  follows  next. 

"Flow"  and  Interactive  Instruction 

Connecting  the  work  of  Csikszentmihalyi  in  the  field  of  motivation  theorizing  to 
computer-based  interactive  instruction  represents  the  final  expository  section  of  this  paper. 
As  can  be  seen  from  the  summary  earlier,  his  work  lies  with  those  researcher/theorists  we 
classify  as  phenomenological  or  holistic  in  their  approaches.  As  such,  his  work  is  probably 
suspect  by  those  who  require  empirically  reductionist  approaches  to  establish  truth.  On  the 
other  hand,  his  methodology,  while  subjective,  involves  and  reports  on  people  and  events  in 
the  real  world:  rock  climbers,  chess  players,  dancers,  surgeons,  and  the  like. 
Csikszentmihalyi  (1975)  is  interested  in  why  they  do  (and  keep  on  doing)  the  things  they 
enjoy.  What  is  there  about  the  experience  that  keeps  them  engaged?  His  answer,  "flow,"  is 
one  of  those  words  whose  meaning,  in  this  psychological  context,  is  almost  immediately 
understandable.  We  know  what  it  is  because  we  have  experienced  it  ourselves  and  it 
represents  truth  to  us  as  much  as  any  empirical  result  from  the  lab. 

But  how  can  his  theorizing  provide  guidance  to  designers  of  computer-based 
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interactive  instruction'.'  Malum*  ( lumps  Csiks/ontmihalyi  with  others  who  him*  meeting 
ehullniige  as  the  vital  element  in  intrinsic  motivation  theory  and  faults  him  tor  ignoring  the 
role  of  nirioKity.  ( )thers  (Keller,  1UH7)  fault  his  work  for  being  over  hroad  and  merely 
descriptive.  This  writer  searched  in  vain  for  "curiosity"  or  "exploratory  behavior"  in  the1 
various  indices  of  ('sikszentmihalyi  s  work  read  for  this  paper.  The  answer,  I  think,  lies  not 
with  hiH  work  or  theories,  but  with  our  own  approaches  to  learning  and  instruction, 

At  its  most  basic,  Csikszentmihalyi's  "flow"  state  is  simply  a  description  of  people 
enjoying  themselves.  They  are  in  a  state  of  enjoyment  because  they  have  situated 
themselves  in  an  environment  that  challenges  them  optimally.  This  should  strike  a 
responsive  chord  within  the  educator  that  should  resonate,  "That's  how  I  want  my  students 
(and  me!)  to  be  in  the  classroom."  It  represents  a  desired  state  to  strive  for  (i.e.,  an 
instructional  goal!)  for  us  as  creators  of  educational  environments.  He  has  thoroughly 
documented  his  extensive  work  with  many  different  populations,  even  across  cultures 
(Csikszc.itmihalyi,  1990a).  That  flow  states  exist  and  are  achieved  regularly  by  people  in 
all  walks  of  life,  at  all  times  of  dayf  at  every  age  he  has  chosen  to  investigate,  is  well 
documented.  If  educators  find  it  difficult  to  see  possibilities  for  enjoyment  in  learning,  flow 
states  will  seem  foreign  through  no  fault  of  Csikszentmihalyi's. 

Besides  being  a  particularly  well  described  goal  for  educators  (including  C  J 
designers),  flow  also  has  the  desired  property  of  being  "autotelic,"  i.e.,  it  seems  to  reward 
itself.  Much  has  been  written  about  the  desirability  of  intrinsic  motivation  vs.  extrinsic 
motivation  in  learning  that  indicates  its  significance  to  educators  (Day,  et  ah,  1971;  Lepper 
&  Greene,  1978).  Csikszentmihalyi's  work  is  so  representative  of  the  field  of  intrinsic 
motivation  his  contributions  have  nearly  become  lost  in  it.  Certainly  Malone  (and  later 
Lepper  and  Malone)  seemed  to  ignore  the  larger  ramifications  of  flow  states  even  as  they 
proposed  motivational  strategies  that  could  likely  bring  them  about.  Perhaps,  again,  there 
is  some  problem  with  seeing  (and  designing)  learning  as  openly  enjoyable. 

Csikszentmihalyi  does  have  much  to  say  about  the  teaching/learning  process  and 
does  so  in  several  of  the  sources  surveyed  for  this  paper.  In  Plav  and  Learning 
Csikszentmihalyi  (1979)  presented  his  flow  theory  to  a  group  of  early  childhood  scholars 
sharing  views  on  the  role  of  "play"  in  child  development.  While  his  work  up  to  that  time  had 
involved  only  adolescents  and  adults,  his  flow  theories  found  a  ready  and  enthusiastic 
audience.  The  participants  quickly  adopted  "flow"  as  synonymous  with  adult  "play"  and 
applied  it  to  their  own  work  with  children  and  play.  Csikszentmihalyi  cautioned  them 
"about  the  separation  of  play  and  'real  lifer1  (p.269)  making  clear  his  position  that  play 
(children's  flow)  was  life,  just  as  adult  flow  was  a  real  and  genuine  experience. 

The  "Flow  Activities  Room,"  a  project  applying  Csikszentmihalyi's  theories  at 
Indianapolis'  Key  School  (a  magnet  elementary  school),  provides  a  place  for  students  to 
engage  in  and  experience  flow  states  through  "structured  pWy"  and  other  coordinated 
activities  (Cohen,  1991).  In  this  room  students  are  expected  to  engage  in  some  purposeful 
artivity  that  they  enjoy.  After  a  year  long  period  of  observation,  a  study  (Whalen  & 
Csikszentmihalyi,  1991)  showed  that  students  were  able  to  describe  fully  their  flow 
experiences  and  did  so  positively.  They  also  demonstrated  a  marked  preference  for  the  flow 
room  and  its  methods  over  their  classroom  subjects  (also  designed  to  be  intrinsically 
motivating  but,  of  necessity,  more  directed).  The  room  and  its  activities  were  seen  by  the 
students  as  challenging  but  enjoyable.  The  learning  that  occurred  there  can  only  be  related 
on  an  individual  basis,  hut  the  authors  conclude  that  the  experience  was  clearly  rich  in 
individual  and  interpersonal  growth. 

Recently  some  investigators  have*  begun  to  relate  Csikszentmihalyi's  work  to  human- 
computer  interactions  (Webster,  Trevino,  &  Hyan,  1993).  Studying  the  "playfulness"  of 
students  and  (later)  employees  using  various  application  software,  the  authors  developed  a 
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measure  of  flow.  They  found  flow,  in  this  context,  had  three  dimensions:  control,  attention 
focus,  and  "enjoyment."  The  latter  was  a  combination  of  curiosity  and  intrinsic  interest. 

In  an  essay  meant  to  provide  philosophical  direction  to  educational  reform, 
Csikszentmihalyi  (1990b)  states  that  "It  is  not  that  |U.  S.J  students  cannot  learn;  it  is  that 
they  do  not  wish  to  "  (p.  115).  His  thesis  is  that  technology  and  methodology  will  not 
educate  students  if  they  are  not  motivated  to  learn.  After  explaining  flow,  Csikszentmihalyi 
maintains  that  students  can  and  do  experience  this  state  on  occasion,  and  that  ediicrtors 
need  to  actively  promote  its  presence  during  instruction. 

Since  it  is  a  naturally  occurring  state  in  someone  engrossed  in  what  they  are  doing, 
teachers  must  be  wary  imposing  conditions  that  tend  to  discourage  its  occurrence.  Citing 
Amabile  (1983),  Csikszentmihalyi  (1991)  says  these  conditions  include:  (1)  excessive 
control,  rules,  procedures,  time  constraints;  (2)  emphasis  on  evaluation,  rewards  and 
punishments,  norm  referenced  competition;  (3)  too  much  emphasis  on  "winning;"  (4)  making 
the  individual  self-conscious.  These  are  all  counter  motivational,  generally,  and  serve  to 
reduce  the  exercise  of  the  individual's  intrinsic  motivators  and  the  likelihood  of  "flow"  states 
occurring  in  the  classroom.  He  concludes,  "It  is  hoped  that  with  time  the  realization  that 
children  are  not  miniature  computing  machines  will  take  root  in  educational  circles,  and 
more  attention  will  be  paid  to  motivational  issues"  (p.  138), 

Conclusion 

The  goal  of  this  paper  was  two  fold:  Place  the  fuller  version  of  curiosity  theorizing 
into  current  thought  regarding  motivation  for  learning,  particularly  as  is  applies  to  the 
creation  of  computer-based  interactive  instruction;  and  relate  these  theories  to  the  "flow" 
state  theorizing  of  Csikszentmihalyi.  With  respect  to  "curiosity"  the  result  has  been  to  raise 
more  questions  than  were  answered.  The  death  of  Berlyne  left  the  field  without  anyone  to 
be  the  grand  visionary.  The  result  has  been  the  near  total  subsumption  of  curiosity  by  the 
motivational  psychologists,  particularly  those  concerned  with  intrinsic  motivation.  Where 
intrinsic  motivation  and  learning  are  discussed,  curiosity  and  motivation  become  nearly  one 
and  the  same. 

Perhaps  this  is  the  best  stance  for  those  interested  less  in  the  (presently 
unknowable)  biological  mechanisms  underlying  cognition  and  impatient  to  get  on  with 
creating  interactive  learning  environments.  While  cognitive  science  and  neurobiology  forge 
links  and  much  is  promised  (Sejnowski  &  Churchland,  1989),  we  are  left  facing  our  students 
today.  As  reflective  practitioners  concerned  with  the  whole  person  before  us,  we  are 
prepared  to  incorporate  as  many  useful  "tricks  of  the  trade"  as  are  needed  to  foster  learning. 
It  is  precisely  at  this  viewpoint  that  Csikszentmihalyi's  flow  state  theorizing  becomes  both  a 
metaphor  and  a  practical  design  goal  for  the  types  of  instruction  that  will  encourage 
learning,  empowering  us  and  our  students  to  better  understand  each  other  and  our  world. 
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Abstract:  As  a  result  of  [the  society]  moving  into  the  information  age, 
changes  in  the  educational  process  as  well  in  the  workplace  arc  inevitable. 
Parents,  students,  school  officials  and  business  leaders  are  currently 
experiencing  the  effects  of  the  information  explosion  era  on  a  daily  basis. 
Upgrades  and  new  releases  of  computer  hardware  and  software  are 
occurring  almost  before  we  can  fully  understand  our  old  systems.  Are 
decision  makers  ready  to  make  the  changes  necessary  to  prepare  student h 
with  the  competency  and  foundation  skills  (as  identified  by  the  Secretary's 
Commission  on  Achieving  Necessary  Skills)  that  they  will  need  for  the 
changing  workplace?  This  paper  will  address  two  trends  in  the 
instructional  technology  field:  reforming  the  educational  process  and 
implementing  electronic  performance  support  systems  into  the  workplace. 
These  trends  are  important  because  they  have  influence  on  moving  the 
teaching  and  learning  process  to  a  new  evolutionary  stage. 
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As  society  approaches  the  21st  century,  technology  will  play  an  important  role  in  the  lives 
of  adults  as  well  as  in  the  lives  of  children.  Business  leaders,  school  officials,  parents,  and 
students  need  to  be  aware  of  the  impact  technological  change  will  have  on  the  educational 
system  as  well  as  on  business  and  industry  (Secretary's  Commission  on  Achieving  Necessary 
Skills  [SCANS],  1991).  Furthermore,  with  markets  becoming  more  global  and  companies 
desperately  striving  to  stay  ahead,  jobs  of  the  future  will  demand  more  highly  skilled 
employees  (Naisbitt,  &  Aburdene,  1985;  SCANS,  1991;  Work  Force  2000,  1988).  During  the 
coming  years,  jobs  that  require  skills  with  information-processing  technology  will  be  of  the  norm 
rather  than  a  rarity  (Naisbitt  &  Aburdene,  1991).  To  prepare  students  for  a  work  force  that  is 
different  from  their  parents  Reigeluth  (1992)  argues  that  the  educational  system  requires 
moving  from  an  industrial  age  model  towards  an  information-age  model  that  views  the 
educational  process  as  the  integration  of  learning  tasks,  with  the  teacher  as  a  coach  or 
facilitator  using  electronic  technologies  as  tools  within  the  classroom. 

The  mastery  of  reading,  writing  and  arithmetic  will  no  longer  be  enough  to  propel  students 
to  success  in  the  work  place.  According  to  the  SCANS  (1991)  report,  researchers  identified  five 
competency  skill  areas  and  three  foundation  skill  areas  that  are  necessary  for  success  in  the 
workplace.  The  five  competency  skills,  as  defined  by  SCANS  (1991)  are: 

1.  Resources:  Identifies,  organizes,  plans,  and  allocates  resources, 

2.  Interpersonal:  Works  with  others, 

3.  Information:  Acquires  and  uses  information, 

4.  Systems:  Understands  complex  interrelationships  and, 

5.  Technology:  Works  with  a  variety  of  technologies. 

In  addition,  included  in  the  SCANS  (1991)  report  is  a  description  of  the  three-part  foundation 
skill  requirement.  They  are: 

1.  "  Basic  Skills:  Reads,  writes,  performs  arithmetic  and  mathematical  operations,  listens  and 
speaks, 

2.  Thinking  Skills:  Thinks  creatively,  makes  decisions,  solves  problems,  knows  how  to  learn 
and 

3.  Personal  Quality  Skills:  Displays  responsibility,  self-esteem,  sociability,  self- 
management,  integrity  and  honesty. 

Will  high  school  or  college  graduates  be  able  to  master  tho  competency  and  foundation  skills 
upon  graduation?  This  question  focuses  on  how  society  is  changing  and  as  a  result  the  needs  of 
students  who  are  preparing  to  enter  the  workforce  have  changed. 


Education  is  our  passport  to  tin?  future,  lor  tomorrow  belongs  to  the4  people  who 
prepare  for  it  today. 

Malcolm  X 

Educators  are  seeking  new  ways  to  ensure  that  students  have  the  appropriate  skills  on  a  local 
and  national  level.  On  the  local  level,  school  officials  are  re-thinking  the  design  of  schools  for 
the  information  age  (Molenda,  1992;  Keigeluth,  1987)  and  on  the  national  level  the  Bush 
administration  announced  America  2000,  a  new  education  strategy,  designed  to  provide 
national  educational  goals  for  the  nation's  schools  (Eley,  Foley,  Freedman  &  Scheel,  1992). 
The  fundamental  purpose  for  these  initiatives  is  to  empower  students  to  tackle  the  challenges 
facing  them  in  "the  real-world". 
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Employees  can  no  longer  believe  in  job  security  (Naisbitt  &  Aburdene,  1985)  or  that  a 
college  degree  alone  (without  constantly  upgrading  ones'  skills)  will  propel  him  or  her  in  the 
workplace.  Employment  for  many  people,  currently  and  in  the  future,  requires  a  commitment 
(whether  embraced  by  the  employee  or  imposed  by  the  employer)  to  change  and  life-long 
learning.  Naisbitt  and  Aburdene  (1985)  argue  that  "in  the  new  information  society  where  the 
only  constant  is  change,  we  can  no  longer  expect  to  get  an  education  and  be  done  with  it.  There 
is  not  one  education,  no  one  skill,  that  lasts  a  lifetime  now"  (p.  141).  Furthermore,  Packer 
(1988)  argues  that  if  the  U.S.  economy  is  to  stay  competitive,  "twenty-five  million  American 
workers  will  need  to  upgrade  their  basic  skills  during  the  1990's"  (p.  8).  This  is  a  low  estimate 
when  taking  into  account  the  bases  for  projections  are  only  on  the  changing  mix  of  jobs  and  do 
not  include  workers  who  need  to  upgrade  their  skills  as  a  result  of  new  technology  (Packer, 
1988). 

Nevertheless,  will  graduates  be  ready  to  face  a  world  that  is  constantly  changing  at  a 
rapid  pace?  There  was  a  time  when  a  clerical  employee  learned  how  to  type  and  file  and  would 
not  have  to  learn  a  new  procedure  for  years-  if  ever.  However,  today  the  replacement  of  a  word 
processor  occurs  almost  as  often  as  the  assignment  of  a  new  manager.  Many  times,  in 
response  to  receiving  this  new  software  application,  the  employee  must  become  proficient 
without  a  decrease  in  productivity.  This  situation  places  the  employee  in  the  position  of 
seeking  the  quickest  and  "least  painful"  method  of  training. 

By  keeping  abreast  of  the  changes  in  society  as  well  as  in  education  and  training, 
instructional  technology  professionals  can  play  a  key  role  in  restructuring  the  educational 
system  from  its  current  industrial  model  to  a  system  that  reflects  the  information  age.  In 
addition,  restructuring  that  includes  integrating  electronic  technology  into  the  educational 
system  can  be  one  solution  for  bridging  the  skill  gap  from  high  school  or  college  to  the 
workplace.  This  restructuring  movement  cn  also  be  the  catalyst  for  using  electronic  technology 
as  a  tool  to  assist  the  teacher  in  developing  the  competency  and  basic  skills  as  defined  by  the 
SCANS  (1991)  report.  Furthermore,  to  bridge  the  skill  gap  for  employees  accessing  new 
information,  employers  are  investigating  electronic  performance  support  systems  (EPSS)  which 
is  a  computer-based  tool  designed  to  assist  the  employed  with  on-demand  training  and 
support. 

The  implementation  of  EPSS  in  the  work  place  and  the  restructuring  of  the  educational 
system  are  two  important  trends  in  the  instructional  technology  field  (Eley  et  al.,  1992;  Seels, 
1993).  These  trends  are  important  because  they  represent  moving  the  teaching  and  learning 
process  in  our  schools  and  on-the-job  to  a  new  evolutionary  stage. 

A  Proactive  Mindset  to  Change 

Systematic  Change  Process 

Change,  whether  planned  or  chaotic,  brinRs  on  stress  and  anxiety.  Moreover,  unplanned 
change  combined  with  minimum  involvement  from  the  people  affected  by  the  change,  produces 
resistance  (Burke,  1982).  Furthermore,  Burke  (1982)  argues  that  "what  people  resist  is  not 
change  but  loss,  or  the  possibility  of  loss"  (p,  52).  This  loss  is  generally  one  of  two  kinds:  loss 
of  the  known  and  tried  or  loss  of  personal  choice  (Burke,  1982).  Rickleman  and  Henk  (1989) 
argue  that  change  does  not  have  to  be  unsettling  if  everyone  involved  take  a  proactive  stand. 

By  taking  a  proactive  stand,  we  shift  from  asking  "What  is  going  to  happen  to  us 
in  the  future?"  to  "What  can  we  do  to  create  the  kind  of  future  we  want?" 
(Rickleman  &  Henk,  1989,  p.  175), 

Kurt  Lewin,  a  psychologist  and  educator,  was  one  of  the  earlier  researchers  to  take  a 
proactive  stance  for  change.  Lewin  argued  that  a  key  principle  to  organizational  change  is  a 
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three-phase  change  process  "unfreezing,  moving,  and  refreezing."  In  trying  to  imagine  the 
phases  required  for  change  Lewin  used  the  metaphor  of  melting  ice.  Unfreezing  means 
reducing  the  negative  forces  through  new  or  disconfirming  information,  which  is  the  function  of 
diagnosis.  Changes  in  attitude,  values,  structure,  feelings,  and  behavior  incorporate  the 
moving  stage  and  refreezing  means  reaching  a  new  status  quo  with  support  mechanisms  to 
maintain  the  desired  behavior  (Weisbord,  1987). 

Many  systematic  change  models  have  evolved  from  Lewin's  model  of  change  (Burke,  82).  A 
key  issue  with  Lewin's  model  is  the  involvement  of  everyone  associated  with  the  process.  In 
addition,  Lewin  states  "people  are  likely  to  modify  their  own  behavior  when  they  participate  in 
problem  analysis  and  solution  and  more  important  they  are  likely  to  carry  out  decisions  they 
have  helped  to  make"  (Weisbord,  1987,  p.  89). 

Educational  reform  is  a  hot  topic  in  the  school  system  as  well  as  in  all  levels  of 
government.  Recently,  the  reform  movement  has  been  shifting  from  a  disjointed  incremental 
effort  towards  more  of  a  Gestalt  perspective,  where  the  focus  of  change  is  on  the  whole 
educational  system  (Bagley  &  Hunter  1992;  Campoy,  1992;  David,  1991;  Pearlman,  1989; 
Raywid  1990;  Sheingold,  1991). 

David  (1991)  argues  that  "restructuring  presumes  the  goal  for  educational  system  is  not 
simply  to  catch  up  to  the  world,  it  needs  the  capacity  to  continue  to  evolve  as  the  world 
continues  its  rapid  pace  of  change"  (p.  77).  To  reach  this  goal  Reigeluth  (1992)  identifies 
"Ernest  Boyer  (1983),  John  Goodlad  (1984),  Theodore  Sizer  (1984),  Lewis  Pearlman  (1987), 
Ann  Lieberrman  and  Lynne  Miller  (1990),  Albert  Shanker  (1990)  and  Banathy  (1991,  1992)" 
(p.  1)  as  advocates  for  systemic  change  in  education.  Reigeluth  (1992)  further  argues  that  true 
educational  reform  has  occurred  only  once  in  our  nation's  history-  when  the  industrial 
assembly-line  model,  which  is  currently  in  place  today,  replaced  the  one-room  schoolhouse. 
This  paper  will  not  address  all  the  issues  associated  with  restructuring  the  school.  However, 
the  author  will  focus  on  a  small  portion  of  the  educational  reform  movement  that  considers 
integrating  electronic  technology  into  the  classroom  and  the  uses  of  EPSS  as  a  tool  for 
promoting  training  and  support  in  education  or  business  and  industry  work  settings. 

Integrating  Technology  into  the  Educational  Process 

Restructuring  the  educational  system  is  not  an  easy  task.  It  can  be  a  very  complicated 
and  long  term  project  yet,  the  results  of  reform  can  be  fruitful  and  enriching  especially  if 
decision  makers  do  not  view  electronic  hardware  and  software  as  the  sole  answer  for 
entrenched  ills.  A  decision  maker  who  believes  integrating  computer-based  technology  (into  the 
school  system)  requires  simply  supplying  teachers  and  students  with  additional  new  hardware 
and  software  is  focusing  on  a  "device-driven"  learning  environment  versus  a  "student-centered" 
environment.  Bagley  and  Hunter  (1992)  cited  Rockman  who  cautions  "policy  makers  against 
'technohype',  which  he  describes  as  the  efforts  by  advocates  and  vendors  to  sell  technology  as 
the  one  and  only  answer  to  restructuring  the  school  system"  (p.  22).  Sheingold  (1991)  further 
argues  that  "computer-based  technology  has  been  brought  into  schools  during  the  past  decade 
largely  because  the  technology  was  seen  as  important  in  and  of  itself-because  it  was  an 
increasingly  centra*  component  of  the  world  of  adult  work"  (p.  77).  Technology  is  more  than 
just  electronic  devices.  Eley  et  al.  (1992)  define  technology  as  applying  scientific  principles  to 
solve  practical  problems-  a  process  that  deals  with  problem  solving.  This  definition  decreases 
the  emphasis  from  specific  devices  and  places  it  on  solving  the  problem  (Wager,  1993).  For 
example  instructional  design  and  teaching  and  learning  methodologies  fit  the  definition  of 
technologies,  however  they  are  forgotten  examples.  The  author  suggests  that  the  lessening  of 
"technohype"  can  occur  by  informing  decision  makers  that  electronic  technology  is  "a  systematic 
blend  of  people,  materials,  methods  and  machines"  (Eley  et  al.,  1992,  p.  27). 
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More  important,  the  integration  of  computer-based  technology  into  the  educational  process 
requires  decision-makers  to  re-think  their  opinions  on  several  topics-  tor  example  their  views 
on: 

1 .  The  role  of  the  teacher  in  the  classroom  as  well  as  their  views  on  teaching  and  learning, 

2.  The  organization  of  student  desks  and  computers  in  the  classroom, 

3.  The  life-cycle  and  maintenance  of  computer  hardware  and  software. 

In  addition  Collins  (1991)  identified,  from  the  literature  and  observations  in  schools  where 
teachers  are  using  computers,  several  major  trends  on  how  computer  technologies  have  an 
affect  on  the  classroom.  First,  Collins  (1991)  argues  that  the  role  of  the  teacher  will  move  from 
a  lecturer  to  a  facilitator  and  coach  who  actively  engages  students  in  long-term  computer 
projects.  These  projects  may  simulate  relevant  "real-world"  problems,  where  students  are 
actively  participating  in  arriving  at  solutions. 

The  classroom  will  shift  from  desks  placed  in  rows  where  the  whole  class  participates  to 
desk  arranged  in  small  groups.  This  new  classroom  arrangement,  along  with  the  teacher's 
encouragement,  can  provide  an  environment  where  students  can  focus  on  cooperation  and 
collaboration.  Lastly,  the  author  suggests  that  by  exposing  students  to  electronic  technology 
used  as  a  tool  or  resource  for  the  teacher  students  will  have  an  opportunity  to  practice  the 
foundation  and  competency  skills  as  describe  by  SCANS  (1991). 

Shcingold  (1991)  provides  several  recommendations  that  may  assist  in  the  change  process  that 
includes  integrating  computer  technology.  They  are  as  follows: 

1 .  Bring  technology  and  learning  to  the  same  'table'  when  restructuring  is  being  planned, 

2.  Reconsider  how  technology  is  organized  in  the  district  and  finally, 

3.  Work  towards  teacher  expertise  in  using  a  critical  mass  of  technology. 

Instructional  technologists  are  also  facing  issues  on  how  to  support  employees,  such  as 
teachers,  administrators  and  business  managers,  when  many  school  systems,  universities  and 
businesses  are  facing  huge  budget  cuts  and  downsizing  efforts.  As  a  result  of  these  changes, 
employees  are  being  called  upon  to  "do  more  with  less"  and  decision  makers  are  looking 
towards  instructional  technologists  to  investigate  ways  electronic  technologies  can  increase 
productivity  with  a  leaner  workplace  and  fewer  people  available  for  employee  support. 

Currently,  employees  may  receive  their  formal  training  off-the-job  in  instructor-led  courses 
or  by  computer-mediated  training  (interactive  video,  computer-based  training,  or  a  combination 
of  both),  however,  when  the  employee  returns  to  the  actual  job,  training  and  support  may 
consist  of  human  interaction,  non-centralized  reference  manuals  or  company  documentation. 
Problems  arise  when  employees  need  specialized  training  but  none  is  available  until  several 
weeks  or  months  later.  Uncertainties  in  the  employee's  ability  also  arise  when  he  or  she  is 
trying  to  complete  an  assignment  and  need  specific  support  but  receive  either  too  much 
information,  conflicting  information  or  no  information  at  all  (Brechlin  &  Rossett,  1991). 

Employers  can  no  longer  ignore  employee  training.  Decision  makers  are  slowly  realizing 
that  training  is  a  factor  in  ensuring  that  the  workforce  is  productive.  It  is  only  after  the 
elimination  of  training  programs  that  decision  makers  sometimes  realize  its  importance. 

"Training  is  like  rowing  against  the  current.   Once  you  stop  you  are  dragged 
downstream." 

Jozef  M.M.  Ritzen 

As  a  result  of  these  problems,  decision  makers  are  entertaining  the  idea  of  providing  employees 
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with  software  applications  that  provide  immediate  support  and  on-the-job  training  (OJT)  from 
the  employee's  desktop  computer  at  the  specific  moment  of  need. 

The  development  of  electronic  performance  support  systems  (an  integrated  electronic 
system  that  provides  training  and  support  at  the  moment  of  need  for  the  employee)  is  a  trend 
in  instructional  technology  that  can  bring  training  and  employee  support  to  the  desktop 
computer.  In  addition,  with  the  introduction  and  advancements  of  tools  such  as:  relational 
and  multimedia  databases,  computer-based  training,  expert  systems  and  on-line  references, 

the  foundation  for  developing  electronic  performance  support  systems  (EPSS)  is  in  place. 

i 

What  is  an  EPSS? 

Since  the  recent  introduction  of  EPSS  into  business  and  industry,  leaders  in  the  field  are 
still  trying  to  develop  a  working  definition  associr.  with  this  tool.  Raybould  (1990)  describes 
an  EPSS  as  a  "computer-based  system  that  impro  s  worker  productivity  by  providing  on-the- 
job  access  to  integrated  information  advice  and  learning  experiences"  (p.  4).  Whereas  Gery 
(1991),  a  leader  in  the  field  of  EPSS,  defines  an  EPSS  as  "an  umbrella  concept  that  includes 
any  of  a  variety  of  performance  support  interventions  delivered  on  computer  to  the  worker  on 
the  job  at  the  time  of  need"  (Clark,  1992,  p.  36). 

In  describing  an  EPSS  there  are  two  major  areas  to  consider,  the  content  and  the 
components  of  the  system.  To  identify  the  contents  of  an  EPSS,  Gery  (1991)  uses  the  term 
"Infobase"  which  is  the  collection  of  information  the  employee  will  inquire  into,  access,  or  have 
presented  to  him  or  her  when  accessing  the  system.  For  example,  the  information  located  in  a 
text  relational  database,  multimedia  database,  expert  system  or  on-line  reference  system  are 
the  type  of  data  the  employee  could  select  from  within  an  infobase.  When  manipulating  the 
infobase,  the  user  interacts  with  the  components  of  the  EPSS  that  can  include  the  following: 

A  range  of  support  mechanisms  and  software  tools  including  an  advisory  system  to 
help  in  instructing  or  executing  tasks  and  decision  making,  commercially  available 
software  programs,  organizational  specific  application  software,  special  purpose 
software  utilities  built  especially  for  use  within  the  EPSS  and  other  interactive 
capabilities  (Gery,  1991,  p.  42). 

Components  of  a  complex  EPSS  can,  ideally,  include  a  combination  of  hypermedia  databases, 
expert  systems,  modular  interactive  training,  a  dynamic  maintenance  system,  as  well  as  other 
interactive  software  support  applications,  whereas,  a  basic  EPSS  may  include  only  a  database 
and  an  on-line  help  system  (Geber,  1991).  The  common  feature  found  in  both  a  basic  and 
complex  EPSS  is  the  ability  of  the  application  to  provide  information  to  the  user  at  the  moment 
of  need  (Scales  &  Yang,  1993). 

EPSS  in  Educational  Process 

Researchers  are  investigating  and  developing  EPSS  and  other  support  tools  in  academia 
as  a  way  to  increase  productivity  in  the  workplace.  Merrill  (1993),  Gustafson  (1993),  and  Dick, 
(1993)  are  advocates  for  increasing  productivity  as  a  result  of  decreasing  the  instructional 
development  time  by  using  an  EPSS.  Educators  from  Florida  state  looks  towards  an  EPSS  to 
increase  productivity  by  providing  an  integrated  information  and  learning  system  for  staff 
involved  in  special  education.  This  project  is  part  of  Florida  State  School  year  2000  initiative 
(Huestia,  1993).  In  the  mist  of  companies  as  well  as  academic  downsizing  employees  are 
being  called  upon  to  "do  more  with  less".  To  assist  employees  with  this  change  EPSS  are  being 
designed  to  bridge  the  training  and  support  gap. 


Decision  makers  who  are  not  ready  to  invest  in  an  EPSS,  may  consider  embedded 
interactive  training,  as  a  solution  for  some  OJT  needs.  Embedded  training,  as  defined  by 
Andrews  (1991),  is  the  integration  of  multimedia  training  applications  into  a  workplace  tool. 
EPSS  and  embedded  training  applications  are  moving  the  training  from  out  of  a  classroom  onto 
the  employee's  desktop  computer.  In  to  prepare  for  the  coming  changes  in  workplace  training, 
managers,  software  developers  and  instructional  designers,  will  be  faced  with  seeking  a  clearer 
understanding  of  the  EPSS  development  process.  More  specifically,  decision  makers  must 
addressing  the  issue  of  moving  the  devslopment  of  training  from  the  end  to  an  earlier  software 
development  stage.  Therefore,  EPSS  or  an  embedded  training  application  will  change  the  way 
they  receive  training  on-the-job.  In  addition,  individuals  involved  in  developing  software  must 
work  together  in  re-thinking  how  to  incorporate  the  development  of  training  applications  as 
part  of  the  initial  software  development  process. 

Changes  in  Software  Development  Process 

In  order  to  take  full  advantage  of  trends  such  as  the  computer-base  training  component  of 
an  EPSS,  or  embedded  training,  changes  in  the-software  development  process  are  necessary. 
Traditionally  the  development  of  training  and  support  documentation  have  not  been  an 
integrated  part  of  the  software  development  process.  Howell  (1992)  provides  a  summary  of  six 
methods  used  for  general  software  development  projects.  Of  these  six  techniques,  none 
address  training  and  support  as  part  of  the  development  process.  However,  advances  in 
computer  hardware  and  software  technology  have  made  it  possible  to  integrate  embedded 
training  and  support  within  the  software  development  process  before  the  completion  of  the 
project. 

The  author  suggests  that  to  effectively  develop  the  training  component  of  an  EPSS, 
embedded  training  or  even  non  electronic  support  materials,  the  traditional  software 
engineering  model  should  be  modified.  Figure  1.  displays  a  software  engineering  model  that 
accommodates  the  parallel  development  of  the  application  software  process,  user  interface 
process,  and  has  been  modified  to  incorporate  the  parallel  development  of  training  and  user 
support  materials.  This  abbreviated  software  development  model  is  one  way  to  displays  a 
general  diagram  of  each  of  the  three  development  processes.   It  is  important  to  note  that 
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Figure  1.  Software  development  model  with  training  &  support 


throughout  each  process  efficient  and  effective  ruminuniriitiiin  uuiHt  ciivur  between  rurh 
development  group.  By  moving  the  training  and  support  prnccHH  from  the  end  of  the  tuiftwiire 
development  process  the  user  will  be  able  to  test  a  final  HyKtem  complete  with  training  and 
support  materials.  In  addition,  this  is  one  suggestion  of  a  software  development,  model  tor  an 
EPSS.  Furthermore,  this  change  would  incorporate  the  price  of  training  in  the  overall  cost  of 
the  system  whereby  eliminating  the  purchase  of  training  jih  an  afterthought,  or  as  part  of  the 
software  bundle  that  is  "nice  to  have". 

Summary 

Educational  reform  and  the  development  of  EPSS  are  occurring  as  the  result  of  the  changes 
in  the  skills  and  knowledge  needed  for  the  workplace  today.  Moreover,  the  implementation  of 
educational  reform  and  EPSS  in  instructional  technology  will  require  following  a  systematic 
change  process  that  looks  at  the  entire  organization  for  change  and  not  just  single  components. 
Restructuring  the  educational  system  is  not  an  easy  task  therefore  it  requires  advocates  for 
change  committed  to  an  on-going  and  sometimes  difficult  procesB.  Any  advocate  for  systematic 
change  in  education  must  first  question  him  or  herself  on  their  views  of  teaching  and  learning 
and  the  role  of  electronic  technology  in  the  classroom.  Having  an  advocate  for  change  who  is 
uniformed  could  potentially  promote  technology  as  a  cure  all  versus  as  a  tool  for  supporting 
teachers  in  their  teaching  and  learning  process.  More  important,  to  successfully  integrate 
computer  hardware  and  software  into  the  teaching  and  learning  practice,  advocates  for  change 
must  prepare  teachers  and  administrators  not  in  just  "how"  the  computer  works  but  also  in 
how  the  teacher  and  student  can  learn  "with"  the  compute  using  it  as  a  tool  or  a  resource.  This 
concept  calls  for  a  change  in  the  role  of  the  teacher,  classroom  structure  and  social  structure. 

Instructional  technologists  are  also  being  called  upon  to  develop  EPSS,  The  idea  of 
supporting  an  employee  with  information,  advice  and  training  at  their  specific  moment  of  need 
is  attractive  to  employers  who  are  seeking  to  increase  productivity  in  the  mist  of  downsizing 
and  budget  cuts.  However  EPSS  is  not  a  cure  all  solution  for  every  training  situation  and  does 
not  necessarily  mean  eliminating  additional  training  before  or  after  using  the  system. 

Lastly,  our  society  has  moved  from  the  industrial  era  to  the  information  and  service  age. 
Decision  makers,  whether  in  the  work  place  or  academia,  will  require  involvement  from  their 
entire  organization  to  prepare  our  future  graduates  with  the  appropriate  skills  for  a  changing 
society.  Are  business  leaders,  school  officials,  parents,  and  students,  ready  for  this  challenge? 
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Introduction 


This  paper  examines  the  results  of  a  case  study  of  an  exhibit  of  art  and  artifacts 
designed  for  children.  The  exhibit  was  well  funded  by  several  sponsors  and  housed  in  a 
major  national  art  institute.  While  museums  tend  to  market  their  exhibits  as 
"educational,"  they  tend  not  to  acknowledge  them  as  "instructional."  The  focus  of  the  case 
study  was  to  attempt  to  apply  the  principles  of  instructional  message  design  (Fleming  & 
Levie,  1978,  1993)  to  the  evaluation  of  the  exhibit.  What  follows  is  a  description  of  the 
exhibit  viewed  critically  from  the  perspective  of  these  instructional  design  principles. 

The  exhibit  is  at  the  Art  Institute  of  Chicago  in  the  Kraft  General  Foods  Education 
Center.  The  Art  Institute  of  Chicago  is  a  large,  well-known  art  museum.  It  is  famous  for  its 
excellent  collections,  particularly  its  collection  of  French  impressionist  paintings,  its  fine 
physical  facilities,  and  its  prime  location  in  "The  Loop"  in  downtown  Chicago.  For  many 
years,  the  museum  had  a  facility  for  children  known  as  the  Junior  Museum  which  was 
housed  in  the  basement  of  the  museum.  The  Junior  Museum  closed  in  1990  for  an 
extensive  remodeling,  and  reopened  in  October  of  1992  as  the  Kraft  General  Foods 
Education  Center,  The  new  "Education  Center"  is  an  excellent  facility  with  a  large  area  for 
orientations  and  mural  projects,  an  auditorium,  two  large  studios  for  student  and  family 
workshops,  offices  for  docent  and  education  staff,  a  teacher  resource  center,  a  family 
storytelling  room,  a  "hallway"  gallery,  and  exhibit  preparation  area,  and  a  large  area  for 
exhibits  that  change  periodically.  This  exhibition  area  is  titled  the  Sol  and  Celia 
Hammerman  Gallery.  This  exhibition  space  measures  approximately  forty  by  sixty-five  feet. 
The  overall  Education  Center  is  large,  attractive,  spacious,  nicely  designed;  the  ambiance  of 
the  Education  Center  belies  the  fact  that  it  is  in  the  basement  of  the  Art  Institute. 

The  inaugural  exhibit  in  the  Hammerman  Gallery  is  titled  Art  Inside  Out: 
Exploring  Art  and  Culture  Through  Time.  This  exhibit  is  the  subject  of  the  case  study 
reported  here.  The  exhibit  opened  in  October  of  1992  and  runs  through  October  of 1994. 
The  exhibit  was  in  preparation  for  approximately  two  years.  The  cost  of  the  exhibit  was 
nearly  $250,000.  The  funding  was  provided  by  the  Chicago  Community  Trust,  the  National 
Endowment  for  the  Arts  and  the  Dr.  School  Foundation.  It  should  be  noted  that  as  a 
condition  for  providing  funding  for  the  Art  Inside  Out  exhibit,  contributors  specified  that 
"major"  works  of  art  be  included  in  the  exhibit. 

While  the  case  study  examined  all  aspects  of  the  exhibit,  the  current  paper  will  present 
a  sampling  of  the  various  components.  Included  will  be  textual  elements,  identification 
games,  videos,  art-related  computer  games  and  contextual  environments  that  had  been 
created  to  elaborate  the  major  artifacts  included  in  the  exhibit.  Selected  components  of  one 
of  these  contextual  environments  was  examined  in  depth  to  provide  a  more  holistic  look  at 
how  some  of  the  principles  work  in  tandem  to  shed  light  on  the  quality  of  an  exhibit. 


The  research  methods  used  in  this  case  study  include  observations  of  visitors  to  the 
exhibit,  interviews  with  the  curator,  other  museum  staff,  and  visitors  to  the  exhibit,  and 
analysis  of  all  text  and  components  that  make  up  the  exhibit.  One  of  the  interviews  of 
museum  visitors  was  with  a  curator  from  an  Israeli  children's  museum.  The  observations 
and  analysis  of  exhibit  components  comprise  the  major  data  base  of  this  case  study. 


Methods  Used  in  the  Study 
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The  interviews  with  the  curator  and  other  museum  staff  provide  useful  and  interesting 
insight  into  the  hopes  and  intentions  they  had  for  the  exhibit,  but  the  actual  exhibit  has  to 
be  judged  upon  what  comprises  the  displays  the  visitor  encounters.  Interviews  with  visitors 
also  yielded  interesting  comments,  but  in  general  this  avenue  of  investigation  reflected  the 
cursory  and  superficial  involvement  of  many  visitors  with  this  exhibit.  Observations  of 
visitors  yielded  a  number  of  valuable  insights  into  the  workings  of  the  exhibit.  Observations 
conducted  at  different  times  of  day  or  different  times  of  year  yielded  vastly  different  results. 
Likewise  careful  analysis  of  the  exhibit  components  and  textual  elements  provided  many 
valuable  insights;  this  analysis  would  appear  to  explain  much  of  the  lack  of  visitor 
involvement  with  the  exhibit.  Touring  the  exhibit  with  a  curator  from  another  children's 
museum  confirmed  many  of  the  observations  and  analysis. 

What  the  Exhibit  Designers  Were  Trying  To  Accomplish 

A  brochure  prepared  for  the  opening  of  the  Kraft  General  Foods  Education  Center 
describes  Art  Inside  Out  as  presenting  "12  works  of  art  from  the  African,  ancient  American, 
European,  and  20th-century  art  collections  within  a  museum  environment,  surrounded  by 
alcoves  exploring  the  origin  of  six  of  these  objects  through  learning  games,  interactive 
computers  /  videos,  and  music."  The  curator  of  the  exhibit  and  other  staff  who  worked  on 
the  exhibit  described  the  "objectives"  of  the  exhibit  as  an  attempt  to  present  a  cross-section 
of  the  museum's  collections  as  an  introduction  for  children  and  other  visitors  not  familiar 
with  the  museum.  The  exhibit  was  also  to  address  the  question  of  cultural  diversity  in  an 
"experimental  and  interactive  environment."  "Experimental"  was  defined  as  using 
presentation  techniques  not  typical  of  the  Art  Institute  of  Chicago. 

The  experimental  nature  of  the  exhibit  is  derived  from  the  exhibit's  division  into  the 
"Inside"  section  which  displays  the  artifacts  and  the  "Outside"  section  which  elaborates  on 
the  artifacts  to  place  them  in  a  richer  context  than  is  usually  done  in  art  museum  exhibits. 
This  elaboration  is  accomplished  with  contextual  environments  for  six  of  the  artifacts  and  an 
interactive?,  art-related,  computer  game  that  deals  with  six  of  the  other  artifacts.  (There 
were  originally  12  artifacts  in  the  exhibit,  but  one  was  stolen.)  A  "calendar"  brochure  for  the 
Art  Institute  states,  "This  installation  will  be  enriched  by  an  interactive  environment  - 
involving  a  mix  of  computers,  video,  games,  and  music  -  that  will  help  visitors  to  more  fully 
understand  and  appreciate  art  from  around  the  world."  The  brochure  says  that  the  exhibit 
is  designed  to  explore  the  point  that  "the  meaning  of  an  object  in  its  original  context  often 
may  vary  from  that  evoked  in  a  museum  gallery." 

Brief  Overview  of  the  Exhibit 

When  n  visitor  enters  the  exhibit,  they  are  in  an  area  with  explanatory  text  panels  and 
several  illustrations  to  their  immediate  left,  three  computer  stations  are  to  their  immediate 
right,  and  straight  ahead  are  several  slit-like  windows  through  the  partition  which  sets  off 
this  area  from  the  rest  of  the  exhibit.  These  window  slits  allow  a  view  into  the  central 
gallery  hall  of  this  exhibition  space  -  the  "Inside"  section  -  where  the  11  remaining  artifacts 
displayed  in  the  exhibit  arc  mounted.  At  this  point  the  visitor  is  faced  with  the  decision  to 
turn  left  or  right  in  order  to  go  around  this  partition.  The  presence  of  the  text  provides  a 
minimal  cue  that  a  visitor  should  turn  to  his  or  her  left,  read  the  text  panels  and  then 
proceed  around  the  partition  to  enter  the  "Inside"  gallery  hall.  However,  there  is  no  explicit 
direction  that  a  visitor  should  do  so.  Visitors  were  observed  proceeding  around  the  right  side 
of  the  partition,  thereby  missing  the  explanatory  text,  the  only  advance  organizer  for  the 
exhibit  .  Thus  the  designers  fail  to  establish  a  perceptual  set  for  some  of  the  visitors  to  the 


exhibit.  They  miss  an  opportunity  to  assist  visitors  with  selective  perception  and  to  help 
them  interpret  what  they  are  about  to  see. 

If  the  visitor  proceeds  to  his/her  left  and  reads  the  text  panels,  they  would  be  walking 
around  the  partition  and  into  the  main  gallery-like  section  of  the  exhibition.  This  gallery 
runs  away  from  and  perpendicular  to  the  partition  with  the  slit-like  windows  mentioned 
above.  In  this  gallery  the  11  artifacts  are  displayed  much  like  they  would  have  been 
displayed  in  the  other  public  galleries  in  the  Art  Institute.  The  paintings  are  mounted  on 
walls,  while  a  vase,  sculptures,  and  a  large  oval  dish  are  in  plastic  vitrines  (Plexiglas 
display  cases)  on  pedestals;  one  large  sculpture  is  free-standing  on  a  platform  in  the  center 
of  the  room.  The  lighting  is  low  key  and  each  artifact  is  accompanied  by  a  label  giving  the 
title  of  the  piece,  the  artist  (or  creators),  medium,  place  of  origin  and  date,  as  well  as  the 
acquisition  information  for  that  art  work. 

However,  the  setup  of  this  gallery  is  somewhat  different  from  what  visitors  might  expect 
to  experience  in  the  other  galleries  of  the  museum  because  the  artwork  is  mounted  at  a 
lower  than  normal  height.  In  addition,  each  artwork  has  a  second  label  which  asks  several 
questions  related  to  the  artifact.  These  differences  might  be  seen  as  concessions  to  children, 
in  particular  the  mounting  of  the  artwork  at  a  more  child-friendly  height.  The  artworks 
housed  in  this  gallery  include  a  Greek  bell  krater  vase;  a  Chinese  grave  guardian  beast;  a 
Mayan  royal  emblem  stone;  an  Italian  Renaissance  painting;  a  large,  Italian,  ceramic,  oval 
dish;  a  Japanese,  wood  block  print;  a  French  Impressionist  painting;  an  African,  carved 
wooden  mask;  an  American,  abstract  sculpture;  a  modern  abstract  painting;  and  an 
American,  constructed,  wood  sculpture.  (A  small,  gold,  Code  armadillo  pendant  had  been 
the  twelfth  artifact,  but,  as  previously  noted,  it  was  stolen.) 

Adjacent  to  this  gallery  hall  are  six  alcoves,  or  contextual  environments.  Three 
environments  are  located  off  of  each  side  of  the  gallery.  Each  environment  provides 
additional  information  or  a  context  for  one  of  the  artifacts  in  the  exhibit.  In  many  cases 
there  is  no  overt  connection  drawn  between  the  artifact  which  inspired  a  contextual 
environment  and  the  information  or  constructions  displayed  in  the  environment.  In  other 
words,  the  principles  that  facilitate  the  perception  of  unity  -  such  as  proximity  and  inclusion 
-  have  not  been  applied.  As  a  result,  the  visitor  fails  to  perceive  the  relationship  between 
the  artifact  and  the  contextual  material  and  the  goal  of  providing  greater  understanding 
and,  therefore,  appreciation  of  the  artifact  is  thwarted. 

After  the  visitor  has  toured  this  gallery  hall  and  visited  the  contextual  environments 
which  might  have  interested  them,  they  would  logically  proceed  out  of  the  gallery  by  walking 
around  the  other  side  of  the  slotted  partition  which  they  had  come  around  to  enter  the 
gallery.  In  so  doing,  they  would  pass  two  visitor  comment  books,  which  solicit  opinions 
about  the  exhibit,  a  series  of  questions  about  art  and  museums  displayed  on  the  wall,  and 
three  touch-screen,  computer  terminals,  each  of  which  has  the  same  compliment  of  art- 
related,  computer  games.  These  art-related,  computer  games  provide  more  context  for  the 
five  artifacts  in  the  exhibit  (and  the  missing  armadillo  pendant)  which  were  not 
accompanied  by  contextual  environments. 

Representative  Sample  Exhibit  Components 

Perhaps  the  best  way  to  provide  an  overall  impression  of  the  exhibit  is  to  discuss 
representative  samples  of  each  of  the  exhibit  components.  Many  of  the  same  problems 
appear  over  and  over  again  in  the  exhibit.  When  this  overall  impression  has  been  conveyed, 
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an  in  depth  examination  of  one  of  the  contextual  environments  will  integrate  these  separate 
pieces  into  one  salient  picture  of  this  exhibit  and  its  difficulties. 

The  Use  of  Textual  Elements 

The  textual  elements  used  in  this  exhibit  play  a  more  central  role  than  the  text  used  in 
many  art  exhibits.  There  is  both  more  text  than  is  usually  found  in  an  art  museum  exhibit, 
and  the  text  takes  very  prominent  positions  in  the  design  of  the  exhibit.  The  choices  of 
typeface,  font  size,  ink  color  and  background  colors  are  consonant  with  principles  of 
figure/ground,  legibility,  and  typeface  selection.  Instead  of  the  diminutive  light  gray  print  on 
a  dark  gray  background  found  in  some  exhibits,  much  of  the  text  used  in  the  exhibit  is  a 
large  typeface  in  solid  black  on  plain  white  backgrounds.  Both  the  size  and  the  readability 
of  this  type  are  positive  aspects  of  the  overall  exhibit.  Unfortunately,  the  exhibit  also  makes 
an  attempt  to  use  smaller  sizes  of  type  to  differentiate  levels  of  meaning  in  many  of  the 
textual  elements  of  the  exhibit.  This  strategy  is  not  successful  as  there  is  no  discernible 
reason  or  logic  for  it.  Perception  principles  indicate  that  the  reader's  attention  will  be  drawn 
to  the  change  in  stimulation,  yet  he/she  cannot  make  meaning  of  it.  It  is  an  apparent  cue 
with  no  significance. 

Other  problems  with  textual  elements  are  poor  writing  and  the  use  of  a  number  of  icons 
dispersed  throughout  the  text  in  the  exhibit.  With  regard  to  the  first  problem,  the  writing  is 
frequently  not  well  organized,  not  always  accurate,  does  not  convey  the  information 
necessary  to  understand  the  exhibit,  assumes  too  much  requisite  knowledge,  is  not  designed 
to  lead  the  visitor  through  the  text  in  a  logical  order,  and  is  studded  with  mistakes  of 
grammar,  spelling,  capitalization,  and  punctuation.  The  second  problem  presents  an 
illustration  of  a  non- research  based  use  of  icons  interspersed  in  text.  It  also  presents  an 
example  of  how  the  collaborative  nature  of  exhibit  development  allows  persons,  such  as 
graphic  designers,  to  insert  elements  into  exhibits  for  decorative  purposes  or,  as  in  this  case, 
hecause  they  were  thought  to  be  "fun." 

Tht1  use  of  these  icons  follows  no  apparent  set  of  rules.  The  small,  black  images  are 
about  the  same  size  as  capital  letters  in  the  text  and  are  interjected  literally  between  words 
in  running  text.  The  icons  appear  frequently  in  some  sections  of  text  and  not  at  all  in  other 
sect  ions  of  text.  Icons  also  appear  in  the  several  timelines  which  are  used  throughout  the 
exhibit.  The  icons  sometimes  appear  to  mean  something  and  at  other  times  they  are  totally 
incongruent  with  the  text.  The  reader  is  frequently  left  wondering  why  a  given  word  has  an 
iron  and  another  word  does  not  have  an  icon.  The  icons  in  many  places  appear  to  disrupt 
the  readers  thought  process  ns  i.o/she  tries  to  figure  out  the  meaning  of  the  icons. 

There  are  a  number  of  strange  juxtapositions  of  icons  and  text  in  the  exhibit.  One 
representative  example  follows  some  text  in  the  contextual  environment  devoted  to  the 
Chinese  Grave  Guardian  Beast.  The  text  reads  as  follows: 

To  make  the  earthly  spirit  comfortable,  elaborate  underground 
tombs  were  erented  with  all  the  furnishings  that  one  would  have 
in  this  world.  In  this  way,  a  spirit  would  never  want  to  leave 
the  tomb  and  cause  trouble  for  the  family  of  the  person  who 
had  died. 

This  icon  nppears  to  be  n  coffee  pot.  (!)  There  is  no  discernible  relationship  between  a  coffee 
pot  and  early  Chines  tomb  furnishings  made  explicit  in  this  contextual  environment.  If  there 
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is  a  meaningful  relationship  here  based  upon  historical  traditions,  it  appears  safe  to  say 
that  most  children  would  not  be  able  to  determine  the  meaning  of  the  icon. 

Other  inserted  icons  are  disruptive  not  only  because  they  lack  a  discernible  relationship 
with  the  text,  but  also  because  they  are  signs  in  our  culture,  i.e.,  they  have  conventional 
meanings  incongruent  with  the  passage  where  they  appear.  For  example: 

It  is  only  in  recent  years  that  scholars  have  learned  how  to  read 
the  complex  Mayan  hieroglyphs.  Even  when  the  stone  was  carved, 
most  Mayans  could  not  read  it.  .  .  . 

The  arrow  implies  directionality;  it  means  'look  in  this  direction."  Inserting  this  sign  to  look 
away  must  surely  be  disruptive  to  someone  reading  a  line  of  text,  especially  if  the  reader 
were  a  novice. 

There  is  also  the  repeated  use  of  one  icon  where  it  appears  that  the  icon  has  several 
different  meanings,  some  apparent  and  some  not.  Several  examples  of  text  and  how  this 
icon  is  used  are  presented  below: 

New  ideas  and  inventions  came  one  after  the  other.  People  changed 
the  way  they  thought  about  the  world  after  Nicolas  Copernicus  wrote 
a  book  that  proved  that  the  earth  revolved  around  the  sun.  .  .  . 

.  .  .  This  public 

record  of  a  king's  family  history  told  everyone  that  the  ruler  had 
a  right  to  serve  as  a  link  with  the  gods. 

African  carvers  generally  choose  soft  young  wood  for  masks 
because  it  is  lighter  for  the  wearer  and  easier  to  carve.  They 
carefully  choose  the  tree  to  be  used  and  make  offerings  to  the 
nature  spirits  that  the  Bwa  believe  inhabit  trees,  rocks,  and  water.  .  .  . 

This  icon  (a  dot  with  concentric  circles)  appears  to  be  a  solar  system  which  at  least  bears 
some  relationship  to  the  first  passage  where  it  appears.  However,  its  meaning  in  the  other 
two  passages  is  not  clear  at  all.  Cognitive  psychology  tells  us  that  readers  cannot  help  but 
try  to  make  meaning  out  of  this  confusing  image  in  this  context.  It  is  difficult  to  imagine 
that  such  confusion  could  be  considered  "fun."  One  is  reminded  here  of  the  principles  that 
suggest  that  the  effect  of  humor  in  an  instructional  message  is  unpredictable. 

Identification  Game  in  Grave  Guardian  Beast  Environment 

In  the  contextual  environment  which  accompanies  the  exhibit's  Chinese  Grave 
Guardian  Beast,  there  are  two  glass-covered  rases  set  into  the  wall.  In  each  case  there  are 
several  cardboard,  cut-out  figures  which  represent  camels,  horses,  court  members,  and 
hunters.  These  are  "interactive"  identification  games.  Each  case  has  two  questions  and  two 
buttons  to  push  to  reveal  the  answers  to  the  questions.  These  figures,  which  look  like  poor 
Xerox  copies  which  have  been  enlarged  and  mounted  on  foam  board,  include  excessive  detail, 
making  it  difficult  to  see  their  criterial  attributes.  Principles  regarding  cueing,  visual 
contrast,  and  light  levels  are  violated  in  their  rendering  and  presentation.  Only  one  of  the 
figures  was  described  in  the  text  of  the  exhibit;  for  the  other  three  figures,  the  visitor  must 
rvly  on  the  text  of  the  question  and  his/her  ability  to  decipher  what  the  figures  depict  to  be 
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able  to  answer  the  questions.  An  example  question  would  be: 

Clue:   Hunting  is  my  favorite  pastime.  I  always  hunt 
with  my  specially  trained  bird,  a  falcon.  Who  am  I? 

After  guessing  or  figuring  out  which  is  the  figure  which  is  holding  a  bird  (the  only  clue),  the 
visitor  should  press  the  button  under  the  question  to  reveal  the  answer.  The  rendition  of 
the  figures  is  so  poor  that  it  is  difficult  to  see  which  figure  is  holding  a  bird.  The  answer  is 
revealed  by  a  light  which  comes  on  under  the  correct  figure.  This  is  the  level  of  interactivity 
in  much  of  the  exhibit,  with  the  exception  of  the  art- related  computer  games  which  will  be 
discussed  later.  Answers  to  the  questions  asked  either  rely  on  superficial  characteristics  of 
the  objects,  requiring  no  problem  solving  and  no  inference,  or  they  rely  on  prerequisite 
information  that  most  visitors  do  not  have  in  their  existing  schemata.  In  other  words,  the 
questions  are  either  too  easy  or  too  hard.  This  circumstance  violates  principles  suggesting 
that  attention  and  motivation  are  best  maintained  when  complexity  is  at  a  moderate  level  - 
challenging,  but  doable.  This  game  also  represents  the  significance  level  of  much  of  the 
content  in  the  exhibit's  text  overall.  No  exposition  of  the  meaning,  importance,  or  frequency 
of  falconry  in  China  is  given  in  this  environment.  This  activity  is  meant  to  provide  context  to 
the  Grave  Guardian  Beast  which  is  displayed  at  the  entrance  of  this  contextual 
environment;  however,  the  only  apparent  connection  is  things  Chinese. 

Video  Presentation  in  Butterfly  Mask  Environment 

The  contextual  environment  for  the  Butterfly  Mask  contains  text  about  the  use  of 
masks  among  the  Bwa  people  of  Africa  where  this  mask  originated.  There  is  also  a  display 
of  masks  from  countries  around  the  world  and  text  about  the  use  of  masks  in  some  of  the 
cultures  represented.  There  is  a  stylized  reproduction  of  a  butterfly  mask  and  the  type  of 
costume  that  might  have  been  worn  with  this  type  of  mask.  In  addition,  there  is  a  brief 
video  shown  continuously  in  the  environment,  which  depicts  the  type  of  festival  where  the 
mask  might  have  been  worn  and  the  dancing  which  might  have  been  performed  when  the 
mask  was  worn  in  such  a  festival.  (The  original  Bwa  mask  is  nearly  100  years  old;  the 
video  was  taken  in  recent  times  by  Christopher  Roy  of  the  University  of  Iowa.) 

This  display  of  the  mask,  the  text  describing  the  mask's  role  in  Bwa  culture,  the 
stylized  reproduction,  and  the  video  are  all  intended  to  honor  cultural  diversity  which  was 
one  of  the  goals  for  the  exhibit  and  the  artifacts  chosen  for  inclusion.  The  issue  here  is  what 
message  is  conveyed  to  the  exhibit's  visitors  —  those  who  tour  the  exhibit  superficially  and 
those  who  read  all  the  text  and  watch  the  video  tape.  The  text  and  Christopher  Roy,  the 
videographer,  explain  that  the  Bwa  people  wear  these  masks  and  dance  in  a  frenzy  in 
something  of  a  fertility  rite  to  increase  the  chance  of  having  a  good  harvest.  Dr.  Roy  and  the 
curator  are  both  interested  in  conveying  a  sensitive  picture  of  the  Bwa  people. 

However,  it  is  questionable  that  this  is  accomplished  in  this  exhibit.  The  problem  here  is 
what  communication  theorists  call  "framing,"  i.e.,  the  tendency  for  humans  to  interpret 
stimulation  in  terms  of  their  previous  experiences.  This  process  is  closely  related  to  the 
principles  governing  perceptual  organization  and  response  set.  Most  visitors  to  the  exhibit 
do  not  believe  that  masks  and  dancing  will  help  to  bring  about  an  improved  harvest.  There 
is  a  chance  that  they  will  perceive  the  Bwa  as  "silly.1'  More  superficially,  they  may  associate 
the  frenzied  movements  of  the  dancers  with  child-like  behavior  at  best  and  with  mental' 
disturbance  at  worst.  It  is  questionable  that  enough  context  could  be  provided  in  this 
exhibit  to  allow  naive  visitors  to  refram^  their  perceptions  of  this  video.  The  tragedy  here  is 


that  the  exhibit  designers  may  have  unintentionally  confirmed  prejudices  that  some  visitors 
have  about  the  value  of  African  cultures. 

Art-Related  Computer  Games 

Near  the  front  of  the  exhibition  space,  there  are  three  computer  stations.  These 
computers  each  offer  the  same  games  and  other  activities.  The  games  and  activities  are 
meant  to  provide  additional  context  for  the  six  artifacts  in  the  original  exhibit  that  were  not 
contextualized  with  environments.  These  computer  stations  are  fairly  sophisticated.  They 
are  equipped  with  touch  screens  which  allow  players  to  access  all  parts  of  the  games  -  there 
is  not  a  keyboard  or  mouse  in  sight.  The  games  which  operate  on  these  computers  utilize 
brilliant  color,  animation,  fading,  and  sound,  and  the  player  has  the  ability  to  move  or  drag 
various  images  around  the  screen  during  some  of  the  games.  For  the  most  part,  the 
programming  is  competent,  but  the  content  is  quite  limited.  These  machines  could  do  much 
more  if  the  games'  design  had  boon  more  sophisticated  or  perhaps  was  overtly  instructional. 

There  are  eight  different  games  on  each  computer,  as  well  as,  "visits"  to  several  artists' 
studios,  and  a  look  at  three  additional  paintings.  The  computer  games  make  more  use  of 
the  technology,  while  the  two  activities  are  basically  slide  shows.  The  computer  games 
allow  players  to  create  a  mock-up  of  a  ceramic  dish,  play  a  decorative  arts  matching  game, 
go  on  a  treasure  hunt,  remember  the  colors  in  a  painting,  change  the  color  scheme  of  a 
painting,  construct  a  sculpture  from  a  collection  of  scrap  metal,  deconstruct  a  sculpture  and 
reconstruct  it,  and  play  "brushstroke*  detective,"  a  game  that  allows  matching  of  paintings 
by  identification  of  brushstroke  technique.  An  explanation  of  two  of  these  games  will  serve 
to  illustrate  the  sophistication  of  the  games  and  their  utilization  of  the  technology  . 

In  the  Brushstroke  Detective  game,  the  players  are  given  sections  of  four  paintings  and 
four  smaller  circular  portions  of  these  same  paintings.  The  instructions  tell  the  players  to 
match  the  circular  portions  to  the  larger  sections  of  the  paintings  by  noting  the  type  of 
brushstrokes  used  in  each  painting  and  the  circular  sections.  The  player  then  drags  the 
circular  section  to  the  larger  portion  of  the  painting  by  placing  his/her  finger  on  the  screen 
and  dragging  the  circle  to  where  it  fits  into  the  larger  part  of  the  painting.  If  the  circle  is 
"released"  in  the  correct  position,  the  computer  reveals  the  entire  painting  to  the  game 
player. 

The  major  flaw  in  the  conceptualization  of  this  game  is  that  the  brushstrokes  are  not 
the  most  apparent  characteristic  of  the  illustrations  given.  Frequently,  the  more  salient 
characteristics  are  color  and  pattern.  The  learner  cues  on  these  characteristics  rather  than 
the  type  of  brushstrokes  that  were  used,  thus  circumventing  the  objective  of  the  game.  The 
designers  of  the  game  have  done  a  poor  job  of  identifying  the  cognitive  processes  actually 
required  by  the  game.  Therefore,  the  objective  of  the  game  and  the  process  of  playing  it  are 
misaligned. 

Another  of  the  games  is  described  as  a  "decorative  arts  matching  game,"  In  this  game, 
the  player  is  given  a  picture  of  a  small  portion  of  some  decorative  arts  object  and  the  choice 
of  "matching"  this  picture  with  the  names  of  four  decorative  arts  objects:  bowl,  clock,  vase, 
or  piano.  The  player  makes  the  match  by  touching  the  name  of  the  object  which  he/she 
thinks  matches  the  small  portion  of  the  object  depicted.  There  are  always  the  same  four 
names  from  which  to  choose. 

The  objective  of  this  game  is  not  to  match,  but  to  guess.  The  player  has  not  previously 
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been  given  any  information  by  the  computer  or  in  the  overall  exhibit  about  any  of  the 
decorative  arts  objects  depicted  in  the  game  .  Indeed  there  is  even  very  little  information 
contained  in  the  small  pictures  from  which  one  might  make  an  educated  guess.  This 
experience  is  very  likely  not  to  be  a  rewarding  one  for  the  visitor.  They  are  set  up  to  fail  at 
this  matching  task.  Here  again  the  game  designers  seem  to  have  done  a  poor  job  of 
analyzing  both  the  cognitive  processes  required  by  the  game  and  the  prerequisite  knowledge 
required  to  play  it.  Therefore,  the  game  does  little  to  add  understanding  or  appreciation  of 
the  larger  exhibit. 

Representative  Contextual  Environment: 
The  Adoration  of  the  Magi 

In  order  to  describe  how  several  of  these  components  are  integrated  in  a  single 
contextual  environment,  selected  elements  in  the  environment  for  the  painting  The  Adoration 
of  the  Magi  is  examined  and  criticized  in  some  detail  below. 

Original  Painting  and  Labels 

The  painting  The  Adoration  of  the  Magi  is  hung  in  the  "Inside"  gallery  section  of  the 
exhibit  Art  Inside  Out.  The  painting  is  accompanied  by  two  labels.  The  first  is  a  typical  art 
museum  label  which  gives  the  title  of  the  painting,  the  artist,  the  medium  (tempera  on 
panel),  geographic  origin  (Italy),  approximate  date  of  its  creation,  and  the  persons  donating 
the  work  and  the  accession  date.  This  first  label  reads  as  follows: 


THE  ADORATION 
OF  THE  MAGI 

Attributed  to  Raffaelo  Botticini 
Tempera  on  panel 
Italy,  c.  1490 

Mr.  and  Mrs.  Martin  A.  Ryerson  Collection,  1937.997 


The  second  label,  which  asks  four  questions,  is  directed  at  a  cognitive  level  for  very  young 
visitors  /  learners.  This  second  label  asks  the  reader: 


Find  the  three  crowns  of  the  three  Magi. 

How  many  different  animals  can  you  find/ 
How  many  people? 

Do  you  hear  any  music  at  this  event? 
What  kind? 
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This  type  of  label  is  r.Jt  typically  found  in  art  museums.  This  label  is  an  attempt  to  make 
this  exhibit  communicate  more  with  its  visitors,  particularly  to  stimulate  conversation 
between  children  and  parents,  docents,  or  teachers.  However,  finding  crowns,  counting 
people  or  animals,  or  identifying  that  someone  in  a  painting  is  playing  a  musical  instrument 
would  pose  a  challenge  for  only  the  youngest  visitors,  perhaps  first,  second  or  third  graders. 
However,  the  labels  also  pose  a  problem  for  such  visitors  in  that  it  asks  the  visitor  to  "find 
the  three  crowns  of  the  three  Magi."  For  young  visitors,  "Magi"  would  be  a  problematic 
word,  particularly  if  the  child  was  not  of  the  Christian  faith.  It  should  be  noted  that 
diversity  is  not  being  addressed  well  when  those  of  a  non- Christian  background  -  those  not 
familiar  with  the  term  Magi  -  are  excluded.  This  insensitivity  to  those  not  of  the  Christian 
faith  is  demonstrated  in  several  places  in  this  environment.  Here  again  we  see  questions 
that  are  at  once  too  easy  and  too  hard,  both  superficial  and  demanding  prerequisite 
information  not  available  to  many  if  not  most  young  visitors.  Failures  of  both  learner 
analysis  and  task  analysis  are  evidenced  in  these  questions. 

In  the  contextual  environment  which  accompanies  this  painting  there  are  several 
individual  components,  including  a  "viewer"  and  a  model,  two  exhibit  cases  set  into  a  wall,  a 
timeline  and  two  maps  and  a  "big  book"  discussing  the  iconography  of  the  painting. 

The  "Viewer"  and  the  Three-Dimensional  Model 

As  one  enters  the  contextual  environment  adjacent  to  The  Adoration  of  the  Magi,  there 
is  a  "viewer"  consisting  of  a  partition  which  holds  a  line  drawing  on  acetate  and  an  eyepiece 
on  metal  supports  centered  in  front  of  the  drawing.  When  the  visitor  enters  this  contextual 
environment  and  stands  in  front  of  this  viewer,  they  are  in  one  corner  of  what  is  basically  a 
rectangular  room  approximately  12  by  18  feet.  The  viewer  is  designed  to  look  from  the 
entrance  corner  of  the  environment  to  the  opposite  corner  of  the  room  at  a  three-dimensional 
(3D)  model  of  what  might  be  street  scene  in  a  Renaissance  city  -  with  palatial  buildings 
lining  both  sides  of  a  street  and  a  triumphal  arch  astride  the  center  of  the  street  to  the  rear 
of  the  scene.  The  model  is  quite  large,  and  it  is  set  into  a  frame  which  measures 
approximately  five  by  eight  feet.  The  2D  line  drawing  mentioned  above  is  of  this  street 
scene  and  is  approximately  8  inches  by  twelve  inches.  The  metal  supports  hold  the  eyepiece 
approximately  nine  inches  from  this  drawing.  The  eyepiece  of  tho  viewer  is  located  at  the 
assumed  eye-level  of  the  "generic  child"  in  this  child-friendly  exhibit.  For  many  children,  the 
eyepiece  is  too  high  and  for  others  it  is  too  low. 

The  object  of  this  arrangement  is  to  look  through  the  eyepiece  and  align  the  2D  line 
drawing  with  the  3D  model  located  approximately  7  feet  in  front  of  the  viewer.  It  must  be 
assumed  that  the  model  was  intended  to  demonstrate  the  artist's  technique  of  one-point 
perspective,  A  lack  of  specified  objectives  or  explanatory  text  does  not  help  clarify  the 
purpose  of  the  model  for  the  visitor;  however,  the  text  on  the  front  of  the  viewer  directs  the 
viewer  to  find  the  vanishing  point  -  a  term  used  in  one-point  perspective  drawing. 

The  purpose  of  the  viewer  is  not  very  well  realized;  the  nature  of  the  human  eye 
prevents  the  perfect  meshing  of  these  two  "images,"  The  eye  cannot  focus  on  the  plane  of  the 
drawing  (nine  inches  away)  and  the  "plane"  (volume  /  depth)  of  the  model  (seven  feet  away) 
at  the  same  time.  Further,  there  is  no  connection  made  between  the  viewer  and  the  use  of 
perspective  in  The  Adoration.  The  scene  in  the  mo^el  and  drawing  is  similar  to  those 
depicted  in  a  number  of  Renaissance  paintings,  but  it  is  not  at  all  similar  to  the 
architectural  scene  depicted  in  The  Adoration.  And,  while  the  use  of  perspective  does  play  a 
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major  role  in  the  painting,  it  certainly  is  not  the  best  choice  of  paintings  if  one  wishes  to 
show  an  example  to  teach  about  perspective. 

There  is  text  on  the  side  of  the  viewer  you  face  as  you  enter  the  environment  and 
additional  text  on  the  reverse  side.  This  viewer,  and  the  model  in  particular,  attract  a  great 
deal  of  visitor  attention.  As  the  curator  has  noted,  the  model  is  very  attractive  to  visitors  to 
the  exhibit.  The  model  also  represents  a  major  commitment  in  time  and  resources  for  its 
design,  construction  and  installation.  The  viewer  and  model  section  of  this  contextual 
environment  succeeds  on  the  level  of  attractiveness,  size,  and  its  ability  to  generate  visitor 
interest.  However,  it  is  a  failure  and  overall  negative  aspect  of  the  exhibit  due  to  problems 
with  the  theoretical  basis  for  using  this  31)  model. 

The  use  of  one-point  perspective  allows  the  illusion  of  three  dimensions  whUe  depicting 
them  in  two  dimensions.  A  typical  example  of  this  type  of  illusion  is  that  railroad  tracks 
appear  to  meet  in  the  distance,  when  in  reality  they  are  parallel.  In  this  exhibit,  the  model 
depicts,  in  three  dimensions,  how  a  street  scene  appears  to  come  together  in  the  distance. 
This  is  accomplished  by  using  what  might  be  termed  "3D  perspective"  or  "sculptural" 
perspective,  The  effect  is  achieved  by  distorting  the  shapes  of  the  buildings  and  the  plane  of 
the  ground  in  the  model,  i.e.,  by  having  the  plane  of  the  ground  in  the  model  rise  as  it 
recedes  and  the  buildings  get  smaller  and  closer  together  as  they  "move"  toward  the  rear 
and  center  of  the  model.  Thus  the  visitor  is  asked  to  compare  a  2D  drawing  -  using  one- 
point  perspective  •  with  a  3D  perspective  model  (a  model  using  "sculptural"  perspective). 
These  two  forms  of  perspective  are  not  analogous.  What  this  actually  "says"  is  that  the 
railroad  tracks  (or  the  curbs  of  a  city  street)  really  do  come  together  in  the  distance  -  that 
railroad  tracks  are  not  parallel.  This  is,  of  course,  highly  confusing.  This  confusion  is, 
however,  "modified"  by  the  fact  that  the  purpose  and  explanation  of  the  model  are  so  poorly 
presented  that  the  discrepancy  is  not  perceived  by  most  visitors  —  they  do  not  understand, 
but  only  see  the  attractiveness  of  the  model.  The  seriousness  of  this  error  on  the  part  of  the 
exhibit  cannot  be  underestimated. 

The  fundamental  error  of  the  viewer/model  is  difficult  to  explain  in  a  few  words. 
However,  once  the  mistake  in  the  theoretical  grounding  of  this  configuration  of  2D  and  3D 
images  is  conceptualized,  the  seriousness  of  the  error  becomes  apparent.  The  implications 
for  what  is  communicated  to  the  visitor,  the  waste  of  resources,  and  the  loss  of  opportunity 
that  results  from  this  error  are  made  clear.  This  viewer/model  configuration  gives  the  visitor 
an  attractive,  but  highly  confusing,  i.e.,  wrong,  explanation  of  perspective.  Even  if  this 
confusion  only  operates  at  a  subconscious  level,  the  visitor  is  less  likely  to  want  to  return  to 
the  museum.  Confusing  messages  violate  recently  articulated  principles  of  maintaining 
motivation;  humans  are  motivated  to  attend  to  information  that  is  challenging,  but  not 
incoherent  or  internally  irreconcilable. 

Wall-Mounted  Display  Case  About  Egg  Tempera  Paint 

If  a  visitor  continued  to  proceed  around  the  contextual  environment  in  a  clockwise 
fashion,  they  would  next  encounter  a  wall-mounted  display  case  (set  into  the  wall)  which 
addresses  the  egg  tempera  paint  used  by  Botticini  to  paint  Thv  Adoration.  This  display 
case  contains  both  text  and  artifacts,  i.e.,  objects  and  materials  which  illustrate  the  text. 
There  are  a  number  of  problems  with  the  text  and  the  information  it  presents.  As  with 
other  portions  of  the  exhibit,  there  is  no  connection  (delineation  of  objectives)  drawn  between 
The  Adoration  and  this  exhibit  case.  The  visitor  is  left  to  assume  that  this  case  seeks  to 
"explain"  the  tempera  paint  utilized  in  the  painting.  The  visitor  would  have  had  to  draw 


this  inference  from  tLcl  phrase  "tempera  on  panel"  from  the  gallery-like  label  which 
accompanies  the  painting  in  the  "inside"  gallery  portion  of  the  exhibit.  This,  of  course, 
assumes  the  visitor  read  the  label.  (Many  gallery  visitors  do  not  read  labels  due  to  their 
small  size  and  the  dearth  of  information  they  give  -  and  possibly  because  the  labels  are 
confusing.)  Observations  of  visitors  to  the  exhibit  reveal  that  many  visitors  do  not  read  the 
labels  which  accompany  The  Adoration. 

A  major  problem  with  this  display  case  is  the  text  is  poorly  written  and  not  particularly 
accurate.  For  example,  the  statement,  "A  layer  of  rabbit  skin  glue  and  chalk  gesso,  a 
plaster  of  paris  [sic]  mixture  [sic]  are  applied  to  prevent  the  wood  from  deteriorating,"  is 
fraught  with  problems.  Of  course,  the  word  "paris"  should  be  capitalized.  It  also  appears 
that  this  sentence  is  jumbled.  Perhaps  the  curator  intended  to  say,  "Renaissance  painters 
painted  on  wood.  A  layer  of  chalk  gesso  (a  mixture  of  rabbit  skin  glue  and  plaster  of  Paris) 
was  applied  to  prevent  the  wood  surface  from  deteriorating"  As  the  text  was  originally 
written,  a  novice  visitor  might  find  themselves  asking  "What  wood?"  Even  if  the  visitor  read 
the  label  on  the  original  painting  and  recalled  the  wording  on  the  label,  the  label  says 
"panel,"  not  "wood."  Furthermore,  even  if  the  sentence  is  clarified  and  "unjumbled"  for  the 
visitor,  the  novice  is  misinformed.  The  purpose  of  gesso  is  not  to  protect  the  wood,  but  to 
provide  a  smooth  white  surface  on  which  the  tempera  paint  might  be  applied. 

Overall,  this  wall  display  case  is  confusing.  It  had  the  potential  to  explain  an 
important  element  (tempera  paint)  in  the  creation  of  The  Adoration.  The  important 
potential  of  the  display  case  is  particularly  undermined  by  the  lack  of  a  clear  connection 
between  the  tempera  paint  medium  and  the  painting,  The  Adoration  of  the  Magi,  which  is 
the  "excuse"  for  the  introduction  of  the  topic  of  tempera  paint  into  this  contextual 
environment.  The  text  in  the  display  case  violates  principles  regarding  the  use  of  standard, 
correct  English  as  well  as  principles  of  structured  text  suggesting  continuity  in  terminology. 

Time  Line 

Proceeding  around  the  room,  the  visitor  would  encounter  a  time  line.  A  similar  but 
somewhat  more  extensive  time  line  was  also  part  of  the  introductory  section  for  the  entire 
exhibit.  However,  the  time  line  in  this  contextual  environment  has  a  section  of  the  "line" 
colored  in  brown,  900  to  1900  AD.  There  appears  to  be  no  logical,  discernible  reason  why 
this  block  of  time  is  highlighted  in  color.  Perhaps  it  would  have  made  sense  to  highlight  the 
period  of  time  which  is  generally  considered  to  encompass  the  Renaissance.  The  time  line 
uses  color  as  a  cueing  device  -  a  good  perceptual  strategy  -  but  focuses  the  visitor's 
attention  on  a  time  period  that  does  not  represent  the  Renaissance. 

This  time  line  (and  the  six  other  time  lines  in  the  rest  of  the  exhibit)  lists  a  number  of 
historical  "happenings"  and  their  associated  dates.  Some  of  these  historical  happenings  are 
accompanied  by  an  icon  which  might  be  associated  with  the  historical  happening  it 
accompanies;  however,  some  of  these  icons  are  indecipherable.  The  text  used  in  the  time 
line  is  crowded  and  suffers  from  a  lack  of  good  text  design;  the  information  included  for  the 
entries  on  the  time  line  is  inconsistent  and  its  placement  varies  from  entry  to  entry.  .  Very 
few  visitors  have  been  observed  to  spend  a  significant  or  even  a  small  amount  of  time 
viewing  the  time  line  in  this  environment  (or  those  elsewhere  in  the  exhibit).  This  may  be 
due  in  part  because  these  significant  historical  happenings  are  not  explained  or  related, 
except  by  chronology,  to  The  Adoration  and  because  the  time  line  overloads  perceptual 
capacity. 
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Recommendations  for  Improving  Museum 
Exhibit  Development  Practices 


While  the  principles  discussed  in  this  case  study  are  generally  accepted  by  instructional 
designers,  t  hey  run  counter  to  the  thinking  of  and  are  not  a  part  of  the  knowledge  base  of 
many  museum  professionals.  For  example,  providing  guidance  for  the  visitor  through  the 
exhibit  would  be  counter  to  the  expressed  beliefs  of  the  exhibit's  curator.  In  an  interview  she 
stated,  "An  exhibition  in  a  museum  is  different  from  say  a  course  in  a  classroom.  If  you  are 
teaching  a  class  and  you've  got  their  100%  attention  for  an  hour  or  whatever  you  get,  and 
you  have  specific  goals  and  objectives  and  a  lesson  plan.  An  exhibition  is  a  larger  kind  of 
entity  and  people  approach  it  differently.  And  you  might  say  you  would  number  things,  but 
I  really  wouldn't  want  to  restrict  a  visitor.  I  would  not  want  to  do  an  exhibit  where  I 
numbered  and  gave  people  a  specific  path.  I  think  that  it's  [an  exhibit]  a  more  complex 
process  and  that  learning  can  take  place.  1  guess  I  don't  feel  that  I  really  want  to  have  that 
kind  of  set  goals;  it's  too  narrow." 

The  authors  of  this  case  study  obviously  disagree.  We  are  inclined  to  believe  that 
museum  professionals  have  ignored  fundamental  human  information  processing  principles 
having  to  do  with  perception,  framing,  prerequisite  knowledge,  short  term  memory,  drawing 
inferences,  and  in  general,  making  meaning.  They  would  be  well  advised  to  master 
instructional  design  principles  as  well  as  to  adopt  instructional  development  processes  to 
increase  the  likelihood  that  critical  steps  such  as  envisioning  outcomes,  learner  analysis, 
task  analysis,  message  design,  and  formative  evaluation  are  well  carried  out. 

The  following  recommendations  will  provide  excellent  guidelines  for  the  improvement  of 
the  exhibit  development  process  at  the  Art  Institute  and  museums  in  general.  These 
recommendations  address  systemic  problems  in  the  exhibit  development  process  at  the  Art 
Institute.  The  implementation  of  these  recommendations  will  improve  the  overall 
educational  value  of  exhibits  developed  in  the  future. 

Each  time  a  new  exhibit  is  created,  it  should  have  broad,  general  goals  to  guide  its 
development.  From  these  goals,  specific,  detailed,  precise,  behavioral  objectives  should  be 
written.  These  objectives  should  be  re-written,  added  to,  modified,  expanded,  deleted,  as 
necess  iry,  throughout  the  exhibit  development  process.  Formative  e valuation  (as  well  as, 
the  other  concepts  and  principles  of  instructional  development)  should  be  utilized  to  create 
exhibits   Kxhihits  should  be  evaluated  in  a  mock-up  stage  before  exhibit  elements  and  total 
overall  design  is  finalized.  Problems  discovered  should  be  corrected,  and  the  exhibit  should 
be  evaluated  again,  When  an  exhibit  is  completed  and  on  display,  it  should  be  evaluated 
and  information  learned  should  provide  feedback  to  improve  the  exhibit  evaluated  and  to 
"inform"  the  exhibit  development  process  in  the  future.  It  should  be  noted  that  the 
objectives  nre  what  permit  and  drive  a  meaningful  evaluation  process  -  an  evaluation 
process  that  yields  evidence  about  the  educational  success  of  an  exhibit  and  information 
which  would  improve  future  exhibits  and  the  skills  of  the  exhibit  development  team.  For  the 
optimum  utilization  of  the  feedback  provided  from  the  evaluation  of  an  exhibit,  the 
inf'urmation  gathered  must  be  considored/utilized/reflected  upon  through  the  lens  of  a 
research-based,  instructional  design  schema  or  construct. 

Artifacts  and  works  of  art  utilized  in  an  exhibit  should  be  chosen  for  their  ability  to 
convey  the  objectives  of  the  exhibit.  Use  of  the  '  best"  or  "most  famous"  object  or  wjrk  of  art 
does  not  always  best  facilitate  the  exhibit's  attainment  of  its  objectives.  For  example, 
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Botticini's  The  Adoration  of  the  Magi  might  not  be  the  best  choice  if  a  curator  wanted  to 
illustrate  perspective. 

Instructional  message  design  should  be  utilized  to  facilitate  communication.  The 
decorative  or  "fun"  impulses  of  graphic  artists  should  be  resisted.  Only  decoration  that  does 
not  interfere  with  communication  should  be  allowed.  Graphic  artists  working  in  educational 
settings  should  master  the  principles  of  instructional  message  design.  When  instructional 
message  design  is  utilized  to  create  effective  communication,  there  is  still  much  creative 
work  to  be  done  to  make  an  exhibit,  and  the  included  text,  attractive. 

Spelling,  punctuation,  and  grammar,  should  be  correctly  used.  This  is  especially  true 
when  designing  exhibits  for  children;  the  exhibit  should  model  correct  English.  It  would 
seem  wise  to  hire  professional  proofreaders  for  precision  in  this  matter.  This  is  not  a  call  for 
an  editor  who  might  be  tempted  to  change  the  text  -  thereby  altering  the  meaning  intended 
by  the  instructional  designer/exhibit  developer/curator. 

Unless  a  very  specific  audience  is  the  reason  for  an  exhibit  or  program,  accommodations 
to  one  specific  audience  that  limits  the  access  of  a  number  of  other  audiences  should  not  be 
made.  In  the  case  of  Art  Inside  Out,  the  "child-friendly"  elements  made  the  exhibit  less 
accessible  for  small  or  large  children  and  adults.  Instead,  strategies  that  would  make  an 
exhibit  accessible  to  everyone,  or  the  greatest  number  of  visitors  possible,  should  be  utilized. 
During  the  summer  months  when  school  groups  do  not  visit  the  museum  regularly, 
observations  of  visitors  to  the  Art  Inside  Out  exhibit  indicate  that  the  majority  of  visitors  to 
the  exhibit  are  adults.  This  is  unfortunate  since  a  number  of  aspects  of  the  exhibit  are 
geared  to  the  height  of  a  small  child. 

Instructional  message  design  should  be  mastered  by  museum  exhibit  designers. 
Without  these  principles,  the  exhibit  developer  only  knows  something  is  wrong  and  dues  not 
necessarily  know  why  unless  this  knowledge  is  related  to  a  theoretical  framework  or  schema. 
If  this  theoretical  framework  is  not  present  as  an  "exhibit  development"  schema,  the  curator 
is  likely  to  repeat  the  same  types  of  mistakes  over  and  over  again  in  different  exhibits.  They 
may  never  realize  that  they  are  making  similar  mistakes  which  violate  the  same 
instructional  principles  because  the  mistakes  are  in  a  completely  different  context. 

Incorporate  instructional  design  principles  into  all  facets  of  museum  education, 
including  exhibits,  programs,  teacher  workshops,  and  materials  for  teachers.  For 
educationally  effective  exhibits  and  programs,  it  must  be  recognized  that  some  members  of 
the  exhibit  development  staff  must  possess  knowledge  and  skills  in  instructional  design  - 
skills  which  are  more  important  than  subject  mater  expertise  skills  in  art  history  or  fine 
arts. 
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Introduction 

American  schools  and  the  way  they  teach  have  failed  to  keep  pace  with  information 
technology  changes  in  society  at  large.  Schools  have  clung  to  the  printed  word.  As  a  result, 
students  are  increasingly  distracted  and  disengaged  from  the  formal  learning  process. 
Schooling  fails  to  provide  the  rich,  visual  images  and  interaction  so  readily  available  in  the 
Information  Age.  But  not  all  schools  ignore  the  call  of  technology.  At  the  American  Institute 
for  Learning  (AIL),  a  model  self-help  program  for  youth,  information  is  presented  to  young 
people  in  a  stimulating  way  by  using  interactive  multimedia.  Multimedia  is  typically  defined 
as  the  dynamic  combination  of  technologies  such  as  computer  text,  video,  audio,  graphics, 
and  animation. 

AIL  has  developed  two  drug  awareness  programs  for  secondary  students  who  have 
used  substances  experimentally  or  are  being  challenged  by  peers  to  experiment.  The  first 
program  is  on  addiction  while  the  second  program  explores  the  process  of  recovery.  Two 
separate  production  companies  collaborated  with  AIL  in  developing  these  programs  and  the 
result  is  two  related  programs  with  different  design  and  interface  approaches. 

Program  Overviews 
The  fist  program,  Addiction  and  Its  Processes  is  designed  to  help  young  people 
become  aware  of  the  primary  symptoms  and  phases  of  chemical  addiction  and  the  effects  of 
chemicals  on  the  mind,  body  and  spirit.  The  program  objectives  are  as  follows: 

1.  Identify  phases  of  chemical  abuse  and  addiction. 

2.  Recognize  some  of  the  possible  symptoms  of  each  phase. 

3.  Identify  some  of  the  physiological  and  psychological  effects 
of  each  phase. 

4.  Determine  if  he  or  she  is  in  one  of  the  phases. 

The  power  of  video  illustrates  the  experiences  of  three  young  people  as  they  deal  with  the 
effects  of  their  use  of  alcohol,  marijuana  and  cocaine.  Each  character's  story  is  presented 
separately  through  a  series  of  video  scenes.  Learners  may  follow  each  character  through  the 
phases  of  addiction  linearly  or  select  between  scenes.  Using  the  capabilities  of  the  computer, 
learners  can  explore  issues  and  answer  questions  about  the  video  vignettes.  A  rich, 
informational  database  provides  facts  on  the  characteristics  of  the  phases  of  addiction,  effects 
of  substances  on  the  body,  thoughts  and  feelings  of  the  characters  and  their  friends  and 
family,  and  insight  on  the  character's  perceptions  of  their  addiction. 

The  second  program,  Life  Moves:  The  Process  of  Recovery  shows  that  there  is  hope  for 
recovery  from  chemical  dependency,  what  recovery  involves  and  the  advantages  of  sobriety 
and  a  healthy  lifestyle.  The  following  are  the  overall  program  objectives: 

1.  Identify  the  process  of  recovery  from  chemical  addiction. 

2.  Identify  the  basic  tools  of  recovery. 

3.  Identify  the  critical  role  of  support  groups  in  recovery. 

4.  Identify  actions  necessary  to  maintain  sobriety. 

5.  Describe  the  impact  of  relationships  with  family  and  friends  on  recovery. 
Three  different  young  people's  stories  are  explored  along  with  a  portrayal  of  attitudes  and 
behaviors  of  friends  and  family  that  enable  chemical  dependency.  The  three  characters 
introduced  in  Addiction  and  Its  Processes  are  also  involved  in  the  recovery  process.  This 
program,  using  support  meetings  as  focal  points  intertwined  with  scenes  from  the  character's 
lives,  follows  a  basically  linear  path  through  the  first  year  of  recovery.  The  interactive 
multimedia  allows  people  not  only  to  learn  about  the  process  and  stages  of  recovery,  but  to 
also  witness  and  fell  the  struggles  associated  with  recovery.  The  program  also  provides 
graphical  and  database  information  on  strategies  to  integrate  recovery  into  daily  lives,  the  12 
basic  steps  common  to  many  recovery  methods,  and  material  on  the  nature  of  chemical 
dependency. 
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The  interface  of  these  two  programs  is  different.  Addiction  and  Its  Processes  is 
separated  by  the  three  stages  (early,  middle  and  late)  of  addiction  in  each  of  the  three 
character's  lives.  Learners  aren't  provided  a  recommended  path  and  can  progress  through 
the  program  in  any  manner.  In  Life  Moves  the  lives  of  the  characters  converge  at  a  series  of 
support  group  meetings,  then  diverge  as  they  go  out  into  their  lives.  All  program 
components  are  accessible  through  a  navigational  system  called  the  "Roadmap",  but  several 
features  of  the  interface  encourage  linear  progressions  through  the  story.  At  interactive 
choice  points,  which  are  interspersed  throughout  the  program,  the  interface  discourages 
learners  from  skipping  segments  by  requiring  completion  of  all  branches  before  continuing  on 
with  the  story. 

Evaluations 

AIL  conducted  evaluations  of  Addiction  and  Its  Processes  in  the  Spring  of  1992  and 
Life  Moves  in  the  Spring  of  1993  to  ascertain  if  revisions  were  needed  and  to  evaluate  the 
program's  effectiveness.  Prior  to  the  actual  field  testing,  evaluations  were  conducted  as  part 
of  an  on-going  process  throughout  the  developmental  stages  of  each  program.  When 
developing  costly  and  labor  intensive  interactive  tech  no  logy -based  instruction,  on-going 
evaluation  is  necessity  (Savenye,  1992).  Subject  matter  experts  reviewed  all  scripts  and 
storyboards.  Teenagers  representative  of  the  target  audience  provided  ideas  and  reviews.  A 
common  feature  of  systematically  designed  materials  is  formative  evaluation  which  involves 
testing  draft  versions  of  materials  with  representative  learners  (Andrews  &  Goodson,  1979). 

Formative  evaluation  is  one  of  the  stages  of  the  systematic  design  of  instructional 
materials  (Dick  &  Carey,  1985).  The  three  major  areas  addressed  by  formative  evaluation 
are  : 

1 .  student  achievement. 

2  .  student  and  teacher  attitude  toward  the  content  and  curriculum,, 

3.  Use  of  the  program  in  the  actual  instructional  setting. 
The  evaluations  conducted  addressed  all  three  of  these  important  areas.  Bear  in  mind, 
however,  that  this  evaluations  looks  at  interactive  programs  with  complex  sets  of 
interrelated  factors.  Due  to  the  complexity  of  this  technology,  researchers  are  challenged  to 
isolate  factors  crucial  to  the  success  of  interactive  technologies  (Reeves,  1986;  Savenye,  1990). 

Method 

During  the  Spring  of  1992,  AIL  evaluated  Addiction  and  Its  Processes  in  seven 
different  facilities.  These  locations  represent  the  range  of  people  who  will  actually  use  the 
program.  Six  of  the  locations  were  in  Texas  and  the  seventh  was  in  Arizona.  A  total  of  44 
students  participated  and  used  either  an  IBM  platform  or  a  Macintosh  platform.  Life  Moves 
evaluations  were  conducted  a  year  later  in  five  facilities  with  a  total  of  35  students.  Again, 
students  involved  were  representative  of  the  target  audience.  The  following  research 
questions  were  separately  investigated  for  each  program: 

1 .  How  do  students  use  the  program? 

2.  What  are  the  factors  affecting  students  acceptability  of  the  program? 

3.  How,  effective  is  the  program  in  teaching  content?  What  did  the  students  learn? 

4.  What  are  the  facilitator's  perceptions  of  the  program? 

Instruments 

To  gather  information  about  student  achievement  and  attitudes,  interface  issues, 
screen  design,  authoring,  and  instructional  design  concerns,  data  were  collected  on  several 
different  forms.  Student  data  collected  included  a  general  subject  knowledge  pre-  and 
posttest.  program  posttests  and  attitudinal  questionnaires.  The  students  were  given  the 
pretest  prior  to  any  exposure  to  the  programs  and  then  given  the  same  test  as  a  posttest  after 
they  had  concluded  the  programs.  They  viewed  the  program  and  completed  separate 


posttests  for  each  section.  They  also  responded  to  an  evaluation  survey  (for  biographical  data 
and  for  their  perceptions  on  instructional  presentation  and  content)  as  well  as  an  orally 
administered  student  interview. 

Student  achievement  was  measured  by  the  pre-  and  posttest  and  by  the  posttest  after 
each  section.  The  evaluation  survey  and  student  interviews  assessed  students'  attitudes 
and  perceptions.  These  instruments  were  brief  and  matched  to  the  program's  objectives. 
Prior  to  the  actual  administration  of  these  instruments,  they  were  tested  with  a  small  group 
of  students  to  verify  if  the  tests  were  at  a  seventh-grade  reading  level,  Site  facilitators  also 
reviewed  the  tests  prior  to  their  implementation. 

Data  regarding  the  facilitators'  perceptions  were  gathered  on  an  evaluation 
questionnaire.  On  this  form,  there  were  4-point  scale  from  "strongly  agree"  to  "strongly 
disagree"  questions,  yes/no  response  and  open-ended  questions  to  assess  opinions  on  program 
interface  issues,  content  and  factors  influencing  program  acceptability. 

Data  Analysis 

Program  Use 

Prior  to  using  Addiction  and  Its  Processes,  students  possess  some  information  about 
drugs  and  their  adverse  affects.  Yet  only  eleven  of  the  forty-four  students  were  aware  that 
there  are  three  stages  of  addiction.  When  asked  if  they  thought  it  would  be  helpful  to  have 
an  instructional  program  about  addiction,  all  the  students  agreed.  In  general,  students  came 
to  the  program  open- minded  to  an  instructional  program  on  addiction  and  willing  to  learn. 

With  regards  to  the  question  on  how  students  use  the  program,  the  students  were  in 
agreement  that  the  instructions  and  menus  were  easy  to  use.  When  asked  if  they  felt 
comfortable  using  the  program  without  teacher  assistance,  only  four  students  disagreed. 
However,  it  was  apparent  that  some  students  wanted  to  select  only  the  video  vignettes. 
While  the  students  liked  the  learner  control  and  freedom,  they  may  not  be  well  suited  to 
know  how  to  best  select  instructional  paths.  As  it  is  now,  they  can  skip  relevant 
instructional  content.  Though  the  user  is  encouraged  to  follow  the  story  of  one  addict,  the 
interactive  capabilities  of  random  access  to  other  stories  did  allow  for  a  user  to  get  lost. 
There  is  nothing  within  the  program  that  offers  a  recommended  path  or  that  provides 
information  about  what  to  review  if  remediation  is  needed. 

The  interface  of  Life  Moves  provides  a  basically  linear  approach  by  using  an  interface 
that  links  important  content  to  guide  students  to  program  essentials.  One  student  stated, 
"They  got  to  the  point  and  that  makes  us  think  instead  of  just  watching  the  story."  Another 
student  liked  all  the  different  options,  while  another  felt  that  all  the  choices  made  the 
program  too  long.  Six  of  the  students  found  there  was  too  much  information  presented. 
However,  a  few  students  requested  more  information  on  specific  topics,  such  as  how  to  get 
healthier.  While  the  program  presents  a  comprehensive,  layered  approach,  it  also  requires 
more  learning  time. 

Interestingly,  some  students  watching  Life  Moves  did  not  feel  comfortable  about 
using  the  program  alone.  However,  all  of  the  students  found  the  instructions  and  menus 
easy  to  follow.  The  program  provided  a  command  and  control  center,  but  the  students 
seldom  went  to  it.  When  they  were  led  through  the  program,  the  students  didn't  feel  a  need 
to  reference  a  program  map. 

On  the  Student  Interview  form,  they  were  asked  if  they  liked  using  this  type  of 
program  for  this  topic.  All  but  a  few  of  the  students  said  they  did  with  one  student  noting, 
"It  is  better  than  listening  to  someone  talk."  Five  of  the  students  said  that  it  is  neat 
watching  TV  on  a  computer. 
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Acceptability  Factors 

When  students  were1  asked  if  they  were  influenced  by  Addiction  and  Its  Processes,  all 
but  four  of  the  students  believed  they  had  been.  One  student  said  that  it  was  helpful 
because,  "...it  made  me  think."  The  single  most  engaging  feature  of  this  program  is  the  video 
vignettes.  They  are  powerful,  realistic  and  dramatically  outline  three  young  people's 
chemical  dependency  struggles.  The  stories  were  told  in  short  (2  minutes  or  less)  scenes  that 
clearly  documented  the  stages  of  addiction.  The  students  overwhelming  agreed  that  the 
video  stories  brought  to  life  people's  experiences.  One  student  left  the  room  wrought  with 
emotion  after  viewing  a  scene. 

The  program  dealt  with  numerous  tough  issues  in  a  frank  manner.  For  example,  had 
emotional  or  spiritual  issues  been  avoided,  some  of  the  program's  impact  would  have  been 
lost.  The  students  need  honest  information  in  dealing  with  addiction  and  Addiction  and  Its 
Processes  does  not  skirt  sensitive  issues.  While  you  can  present  information  in  a  realistic 
way  in  a  book,  it  helps  when  you  can  bring  it  to  life  with  video. 

The  video  vignettes  in  Life  Moves  did  not  so  strongly  affect  the  students.  The  stories 
were  presented  and  then  linked  to  additional  content.  The  video  was  in  some  respects  "lost" 
in  all  the  material.  In  fact  one  student  remarked,  "So  much  going  on  that  I  couldn't 
concentrate  sometimes." 

While  a  majority  of  the  student  s  found  it  engaging,  they  did  offer  changes  they 
would  like  to  see  made.  Some  wanted  more  on  the  characters.  It  was  hard  to  get  to  know  the 
video  characters  due  to  the  way  the  program  is  presented.  Several  students  wanted  to 
shorten  the  program  and  to  make  it  more  realistic.  One  student  noted,  "Some  people  may 
find  the  God  parts  disturbing  because  of  the  different  views."  Often  the  students  had  to  click 
on  a  button  to  go  on  in  the  program.  Some  felt  they  were  asked  to  click  to  keep  the  program 
going,  but  they  were  not  allowed  any  instructional  choices.  If  students  are  asked  for  input, 
they  should  be  able  to  make  a  real  choice. 

Effectiveness  Results 

In  general  students  possess  street  knowledge  of  drugs,  but  lack  understanding  about 
medical  and  emotional  drug  issues.  Often  what  they  need  is  a  way  to  sort  out  their  feelings 
and  experiences  concerning  drugs  and  knowledge  to  help  them  differentiate  between  healthy 
and  unhealthy  drug  choices. 

Students  using  Addiction  and  Its  Processes  scored  highest  on  test  questions  that  were 
taught  through  the  video  rather  than  t' Trough  the  informational  sections.  The  emotional 
power  of  the  video  helped  the  students  to  comprehend  the  information.  Also,  the  students 
often  omitted  related  instructional  content,  such  as  information  presented  in  The  Body 
section.  It  appears  that  the  students  gained  new  knowledge  and  were  affected  emotionally. 

Most  of  the  students  using  Life  Moves  could  coiTectly  answer  questions  on  the  video 
segments  on  the  posttests.  Half  of  the  students  experienced  some  difficulty,  however,  with 
Benfs  story.  P;d  this  stem  from  the  complex  issue  of  spirituality  or  from  the  way  the  story  is 
presented?  Th.s  issue  needs  further  exploration. 

Students  were  able  to  correctly  respond  to  questions  covering  content  they  were 
automatically  connected  to  in  the  program,  but  they  did  not  do  as  well  on  topics  they  were 
given  an  option  to  explore.  For  example,  students  had  problems  answers  questions  on 
slogans  presented  in  the  Quick  Help  section.  It  is  probable  that  the  students  never  accessed 
that  section. 


Facilitator  Perceptions 

Facilitators  for  Addiction  and  Its  Processes  found  it  meaningful  for  people  who  are 
trying  to  understand  addictions.  While  they  endorsed  the  program,  they  still  have  some 
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reservations  about  it.  One  felt  that  the  reading  level  was  way  too  high  and  others  felt 
students  would  not  read  the  screens  no  matter  what  the  level.  A  suggestion  was  madt  to 
have  all  printed  screens  read  aloud.  Due  to  the  technical  jargon  presented,  a  read  aloud 
feature  would  be  useful. 

They  gave  a  variety  of  comments.  One  stated,  "This  program  personalizes  addiction 
and  stimulates  an  emotional  response  which  in  turn  leads  them  to  want  to  know  more." 
Another  facilitator,  tentative  in  accepting  this  type  of  instructional  mode  ,  noted,  "I  think 
they  want  to  watch,  but  I  don't  think  they  automatically  learn.  They  have  to  try  to  learn, 
passively  it  won't  happen." 

The  facilitators  found  Life  Moves  easy  to  use  and  informative.  They  all  found  the 
content  accurate  and  without  bias.  Interestingly,  three  of  the  seven  thought  the  program 
should  be  used  after  the  topic  of  recovery  has  been  introduced.  A  broad  introduction  or  a 
chance  to  go  over  prerequisite  knowledge  should  be  included. 

All  the  facilitators  found  that  the  variety  of  resources  were  applicable  and  useful  and 
that  the  information  presented  related  clearly  to  real  life  applications.  However,  there 
appears  to  not  be  enough  opportunities  for  students  to  practice.  Seldom  were  they  given  any 
activities  that  helped  them  practice  or  review  the  content. 

Conclusion 

These  evaluations  were  used  to  validate  and  revise  Addiction  and  Its  Process  and 
Life  Moves.  Both  multimedia  programs  present  stories  enhanced  by  realistic  video  and 
layers  of  textual  and  animated  content.  The  data  indicate  that  these  programs  are  a  viable 
way  to  teach  young  people  about  addiction  and  recovery.  The  power  of  video  pulls  the 
students  into  the  programs,  but  it  can  be  weakened  by  too  many  other  instructional  options. 
In  Addiction  and  Its  Processes,  the  video  was  more  compelling  because  it  was  allowed  to 
carry  the  programs. 

The  results  of  this  study  were  used  to  revise  the  programs,  While  these  specifics  are 
not  presented  in  this  paper,  other  findings  are  of  general  interest.  Designers  should  look  at 
how  to  effectively  integrate  the  video  segments,  ways  to  link  content  that  still  give  learners 
some  control,  how  to  decide  when  student  input  is  truly  needed,  ways  to  teach  technical 
jargon  at  different  reading  levels,  and  how  to  provide  adequate  prerequisite  instruction  in  a 
multimedia  format. 
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Introduction 

Educational  reform  is  a  leading  topic  in  socio  political  debate,  and  was  one  of  the  key 
issues  in  the  latest  U.S.  presidential  election.  Many  educators  envisioned  educational 
technology  (ET)  becoming  a  large  part  of  that  reform.  Ileinich  (1970),  for  example,  foresaw  a 
n»ajor  role  for  educational  technology  as  a  tool  to  support  teachers,  as  a  replacement  for 
teachers  and  a  conduit  for  directly  educating  students,  and/or  as  a  means  to  forming  a 
partnership  with  teachers  whereby  technology  delivers  routine  instruction  and  teachers  focus 
on  planning  and  educational  management.  Indeed,  interest  in  educational  technology  and 
some  practical  successes  during  the  last  50  years  led  educational  technology  to  become  a 
unique  academic  and  professional  field.  The  field  has  academic  departments  and  courses, 
professional  organizations,  journals  and  conferences,  academic  professionals  who  identify 
themselves  as  educational  technologists,  and  of  particular  significance,  a  considerable 
amount  of  scientific  research.  One  could  easily  assume  that  with  so  much  interest  in 
educational  improvements,  with  so  much  potential  for  educational  technology  as  part  of  the 
expanding  information  age  of  technology,  and  with  all  of  the  research  within  the  field  of 
educational  technology,  deep  and  broad  improvements  in  established  education  would  have 
resulted. 

Many  educational  technologists,  however,  lament  what  they  perceive  to  be  few 
implementation  successes  and  a  decHedly  low  impact  of  educational  technology  on 
established  education  (Reigeluth,  1989;  Winn,  1989;  Gentry  and  Csete,  1991;  Heinich, 
1991a).  Educational  technology  is  said  to  be  an  applied  field,  yet  its  knowledge,  based  on 
empirical  research,  is  not  applied  by  practitioners  to  the  degree  expected. 

Many  authors,  publishing  within  the  field  of  educational  technology,  have  analyzed 
the  problem  and  blamed  a  wide  range  of  factors.  Some  view  teachers  themselves  as  the 
culprits;  citing  the  idea  that  teachers  are  threatened  by  perceived  professional  irrelevance 
that  would  cause  them  to  naturally  resist  educational  technology  (e.g.  Heinich,  1991a). 
Other  authors  blame  simple  bureaucratic  inertia  and  lack  of  educational  funding  (Gentry 
and  Csete,  1991).  While  there  have  been  many  accusations  concerning  weak  adoption  of 
educational  technology  in  general  education,  some  educational  technologists  criticize  research 
as  a  malefactor,  it  is  either  too  descriptive  and  not  prescriptive  enough  (Clark,  1989),  it  is 
based  on  too  many  confusing  or  conflicting  theories  (Ross  and  Morrison,  1989),  the  research 
simply  lacks  external  validity  to  everyday  situations  (Reigeluth,  1989),  or  that  it  fails  to 
take  advantage  of  related  research  in  other  fields  (Clark,  1989). 

The  problem  of  innovation  and  adoption  (and  ruminating  self-examination)  is  not, 
however,  unlike  what  occurred  in  the  information  systems  (IS)  field  during  the  early  1980s 
before  the  widespread  proliferation  of  personal  computers  and  readily  available  commercial 
software  packages.  A  considerable  amount  of  information  systems  research  and  writing  has 
been  done  on  who,  what,  when,  where  and  why  (or  why  not)  information  systems  are 
adopted,  including  research  on  why  some  information  systems  are  adopted  but  then  not 
used.  It  is  unlikely  that  the  problems  with  the  adoption  of  educational  technology 
innovations  are  entirely  unique  to  education  and  educational  technology.  Instead  they 
include  problems  commonly  faced  by  proponents  of  any  new  technology. 

ThiH  paper  concentrates  on  the  how  problems  of  educational  technology  adoption.  It 
presumes  thi;  validity  of  educational  technology  research  on  effective  and  efficient 
innovations  and  focuses  instead  on  the  adoption  and  implementation  process  and  it h 
tucturn.  Thin  paper  outlines  and  compares  the  conceptual  definitions  of  educational 
technology  and  information  systems  and  relates  the  histories  of  ET  and  IS  adoption.  It 
outlines  Jind  explains  information  systems  adoption  paradigms,  models,  and  frameworks 
/ind  suggests  similarities  and  differences  between  18  adoption/implementation  and 
educational  change.  Finally ,  this  paper  discusses  educational  change  and  what  can  he 
learned  from  information  systems  adoption  models. 
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Definitions 

One  of  the  immediate  issues  in  discussing  the  educational  technology  adoption 
problem  are  the  varying  definitions  of  educational  technology.  The  Association  for 
Educational  Communications  and  Technology  (AECT)  defines  educational  technology  as 
a  complex,  integrated  process  involving  people,  procedures,  ideas,  devices  and  organization, 
for  analyzing  problems,  and  devising,  implementing,  evaluating  and  managing  solutions  to 
those  problems,  involved  in  all  aspects  of  human  learning.  (AECT,  1977,  p.59) 
Others  define  educational  technology  as  a  methodology  or  set  of  techniques  (Cleary  et  al.v  as 
cited  by  Gentry,  1991),  a  "body  of  knowledge"  (Dieuzeide,  1971,  p.l)  and  as  procedures  and 
devices  (Silverman,  as  cited  by  Gentry,  1991). 

Instructional  technology  (IT),  a  phrase  frequently  used  interchangeably  with 
educational  technology,  often  carries  two  connotations.  The  definition  stated  by  the 
Presidential  Commission  on  Instructional  Technology  (1970)  includes  both  the  view  of 
instructional  technology  as 

the  media  born  of  the  communications  revolution  which  can  be  used  for  instructional 
purposes  along  side  the  teacher,  textbook,  and  blackboard,  (p.  19) 
and  as 

a  systematic  way  of  designing,  carrying  out,  and  evaluating  the  total  process  of  learning  and 
teaching  in  terms  of  specific  objectives,  based  on  research  in  human  learning  and 
communications,  and  employing  a  combination  of  human  and  non  human  resources,  to  bring 
about  more  effective  instruction,  (p.19) 

Engler  (1972)  similarly  defines  instructional  technology  within  two  categories,  the  first  as 
"hardware-television,  motion  pictures,  audio  tapes  and  discs,  textbooks,  blackboards,  and 
so  on1'  and  secondly  as 

a  process  by  means  of  which  we  apply  the  research  findings  of  the  behavioral  sciences  to  the 
problems  of  instruction,  (p.  59) 

One  should  not  view  these  definitions  of  educational  technology  (and  instructional 
technology)  as  nebulous,  contradictory  or  exclusive  definitions,  but  rather  as  inclusive 
definitions  to  bound  the  area  of  interest.  By  combining  tne  essence  of  the  definitions  above 
(and  others  cited  by  Gentry,  1991),  this  paper  defines  educational  technology  as 
the  application  of  people,  devices,  knowledge,  and  procedures  for  efficient  and  effective 
education. 

The  significance  of  combining  the  varying  ET  and  IT  definitions  is  a  clear  correlation 
with  a  textbook  common  definition  of  computer-based  information  systems  (e.g.,  Davis  and 
Olson,  1985;  Laudon  and  Laudon,  1993): 

devices  (computer  hardware,  software  and  communications),  people,  and  procedures  for 
organizing,  storing,  accessing,  and  maintaining  information. 

The  definitions  of  educational  technology  and  information  systems  identically  focused 
on  devices,  people,  knowledge,  and  process  suggest  a  theoretical  linkage  between  ET  and  IS 
application  and  adoption.  The  problems  and  issues  associated  with  adopting  information 
systems  appear  to  have  direct  bearing  on  the  problems  and  issues  with  adopting 
educational  technology.  Information  systems  research  on  adoption,  therefore,  would  seem  to 
offer  rich  insight  and  direction  for  fruitful  educational  technology  adoption  research.  If  the 
theoretical  and  practical  parallels  between  ET  and  IS  adoption  hold,  what  information 
systems  has  learned  about  IS  adoption  may  he  what  educational  technology  can  benefit 
from  in  the  future. 


771) 


Educational  Change:  A  Summarized  History 

Fullan  (1991)  identifies  four  distinct  historical  phases  in  educational  change: 
adoption  (1960s),  implementation  failure  (1970-77),  implementation  success  (1978-82),  and 
intensification  versus  restructuring  (1983-90). 

Fullan's  first  phase  (adoption)  came  largely  as  a  result  of  Soviet  success  in  launching 
a  satellite  in  1957  (years  before  the  U.S.).  The  subsequent  "Sputnik  crisis"  led  to  large-scale 
curriculum  innovations,  technologically-oriented  instructional  systems,  and  the  advocacy  of 
inquiry-oriented  and  student-centered  instruction.  In  the  rush  to  meet  the  crisis,  according 
to  Fullan,  the  emphasis  was  on  how  many  innovations  could  be  adopted,  the  more  the 
better  as  a  mark  of  progress.  During  this  period  instructional  systems  were  researched  and 
developed.  Significant  federal  funding  for  R  &  D  laboratories,  mandated  evaluation  of 
federally  funded  educational  projects,  and  the  redefinition  of  audiovisual  instruction  to 
include  instructional  development  and  technology  gave  the  field  of  educational  technology 
increased  visibility  and  credibility  with  educators. 

During  the  1970s,  however,  innovation  got  a  "bad  name."  According  to  Fullan,  the 
1960s'  innovations  had  been  adopted  haphazardly  with  little  follow -through,  leading  to 
pronounced  implementation  failures.  By  the  end  of  the  1970s,  nevertheless,  there  were 
some  significant,  well-documented  successes  that  provided  important  frameworks  and 
theories  for  comprehensive  educational  reforms.  The  comprehensive  reform  movements  that 
began  in  1983  (as  a  result  of  the  watershed  document  A  Nation  at  Risk  by  the  National 
Commission  on  Excellence  in  Education)  took  many  approaches,  including  the  use  of 
educational  technology. 

The  advent  of  microcomputers  in  the  1980s  appeared  to  offer  the  dawn  of  a  new  era 
with  computer-based  instructional  systems.  The  wide  availability  of  relatively  inexpensive 
desktop  computers,  the  capabilities  of  computer-driven  media,  and  the  inherent  ease  of 
developing,  using,  and  improving  software,  provided  a  ready  vehicle  for  applying  educational 
technology.  By  1989,  76,395  of  the  79,693  U.S.  public  schools  had  two  or  more 
microcomputers,  averaging  about  20  per  school  (Quality  Education  Data,  1989).  Their  use, 
however,  was  primarily  for  administrative  and  clerical  applications  and  not  for  the  process  of 
teaching  and  learning.  The  most  coi.imon  educational  use  of  microcomputers  was  limited  to 
teaching  computer  "literacy"  (Ely,  1991).  Higher  education  wasn't  reported  as  any  better; 
the  average  U.S.  university,  in  terms  of  its  use  of  information  technology  in  teaching,  was 
substantially  behind  the  typical  elementary  and  secondary  school  (Newman,  1989  as  cited 
by  Ely  1991).  * 


The  Information  Systems  Adoption  Problem:  A  Brief  History  and  Comparison  to  the 
Adoption  of  Educational  Technology  within  Education 

Although  electronic  computers  were  used  for  military  purposes  in  the  1940s,  the 
public  application  of  computers  for  information  processing  began  in  1954  when  one  of  the 
first  computers  was  installed  to  process  payroll  at  a  large  U.S.  corporation  (Davis  and 
Olson,  1985).  There  have  been  three  general'*   ^cognized  eras  in  information  systems 
adoption. 

The  first  era  was  from  1954  to  about  1964  when  computers  were  used  for  accounting 
and  clerical  applications  in  major  organizations.  Information  sy stems  were  very  difficult  to 
use  and  expensive.  Very  few  people  understood  how  they  worked,  even  fewer  knew  what  to 
do  with  them.  There  was  a  wealth  of  research  and  theory  that  predicted  enormous  benefits 
from  computers  in  everyday  business  and  personal  life,  far  beyond  accounting  and  clerical 
use,  but  potential  users  and  those  in  management  positions  could  only  wonder  at  the 
futuristic  predictions  while  continuing  traditional  work  habits.  With  considerable 
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simplification,  this  era  could  roughly  equate  to  the  adoption  of  educational  technology 
innovations  in  education  prior  to  1983. 

During  the  second  era,  from  around  1965-1980,  the  breadth  cf  applications 
expanded  due  to  improved  general  purpose  programming  languages.  Major  businesses  saw 
computers  as  a  strategic  weapon,  or  at  least  an  image  maker,  and  management  began  to 
see  the  potential  efficiency  benefits  from  computers.  There  were  large  investments  in 
computers  and  one-of-a-kind  application  software.  Computers  were  ensconced  within  glass 
"throne  rooms"  tended  to  by  computer  specialise  *  who  were  intermediaries  to  users  of 
computed  data.  Users  still  didn't  understand  computers  or  their  potential,  but  they  began 
to  be  exposed  to  the  effects  of  computing.  People  were  mostly  forced  to  adapt  to  computers 
and,  increasingly,  to  depend  on  them  for  record  keeping  as  well  as  finance  and  accounting. 
These  systems  were  designed  by  computer  specialists  who  tended  to  oversell  capabilities, 
had  little  understanding  of  user  needs,  and  increasingly  built  systems  that  either  didn't 
work,  went  way  over  budget,  or  users  wouldn't  use.  Management  perceived  the  importance 
of  computers,  but  not  how  to  apply  them.  As  the  chief  strategist  for  a  major  U.S.  bank  said: 
[Computer]  technology  is  our  top  strategic  concern,  not  because  it  outweighs  everything  else, 
but  because  we  are  unsure  what  to  do  with  it.  Although  we  have  a  strategy  for  the 
marketplace,  the  technology  issues  seem  to  be  eluding  us.  We  can't  seem  to  grasp  the  bigger 
picture  (Parsons,  1983). 

Information  systems  academics  and  professionals  bemoaned  the  dirth  of  effective  IS 
applications  taking  advantage  of  empirical  research,  while  management  complained  that  IS 
research  wasn't  practical  enough  or  relevant 

This  second  IS  era  seems  to  correlate  with  the  present  state  of  educational 
technology  research,  development,  and  adoption.  During  the  1980's,  educational 
technologists  also  foresaw  the  importance  of  the  use  of  technology  in  education.  Educational 
software  increased  in  availability  and  to  become  more  "user-friendly-"  Innovators,  however, 
made  mistakes  similar  to  IS  designers.  Educational  programs  and  products  have  often  been 
designed  by  specialists  who  did  not  understand  the  user  (teacher)  or  the  classroom  learning 
environment.  These  innovations,  therefore,  were  not  implemented  as  the  designers 
intended. 

The  third  IS  era  began  with  the  advent  of  microcomputers  around  1980.  The  entire 
mindset  of  users  adapting  to  computers  was  reversed  as  powerful  applications  that  adapted 
to  users  were  mass-produced  and  made  commercially  available.  Moreover,  non-procedural 
programming  languages  allowed  non-programmers  to  write  software  specifically  tailored  to 
their  needs,  conditions,  and  location.  Simultaneous  communication  innovations  that 
digitally  tied  computers  together  allowed  the  full  potential  (widely  predicted  by  researchers 
in  the  1940s)  of  computers  and  IS  to  overcome  time  and  distance.  For  most  industries, 
information  systems  were  no  longer  a  service  or  simply  a  medium  for  information,  it  had 
become  the  core  impetus  for  an  entire  re-  ngineering  of  organizational  processes.  The  second 
era  issues  about  what  can  be  done  with  information  systems  became  third  era  how  issues 
as  new,  practical  applications  spread.  Information  systems  researchers  began  to  struggle 
just  to  keep  up  with  IS  practice,  let  alone  perform  research  that  isn't  obsolete  before  it's 
published. 

This  third  IS  era  parallels  current  trends  in  educational  technology  in  the  90's. 
Given  the  theoretical  and  practical  parallels  between  educational  technology  and 
information  systems,  educational  technology  should  explore  information  systems  research  on 
adoption  and  implementation  for  insights  and  guidance. 

Information  Systems  Adoption  Theory  and  Research 

Information  systems  research  on  IS  adoption  and  implementation  has  been  ongoing 
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since  the  1950s  with  the  earliest  computer  system  applications.  By  the  1980s, 
implementation  was  one  of  the  four  most  heavily  researched  areas  within  the  discipline  of 
information  systems  [Culnan  and  Swanson,  1986].  Two  basic  adoption  paradigms  were 
used  for  research:  factors  and  process. 

Factor  Paradigm.  The  factor  paradigm,  the  dominant  paradigm  in  information 
systems  implementation  research,  sought  to  identify  and  relate  the  many  factors  involved  in 
IS  implementation  success,  the  what  behind  successful  adoption.  Six  key  variables  have 
been  identified  from  scores  of  empirical  research  and  analysis  efforts: 

1.  organizational  need  and  support 

2.  user  personal  stake  in  success 

3.  user  assessment  of  system  and  organizational  support  for  it 

4.  user  acceptance  of  system 

5.  use  of  system 

6.  satisfaction  [Lucas,  Ginzberg  and  Schultz,  1990]. 

These  factors  are  linked  into  a  generic  model  for  IS  implementation  as  shown  in  Figure  1 
found  in  Appendix  F. 

In  this  model,  management  support  for  a  system,  organizational  changes  required 
as  a  result  of  the  system,  and  the  urgency  of  the  problem  the  system  is  supposed  to  address 
combine  to  affect  the  user's  perceived  stake  in  the  system's  adoption.  User  stake,  in  turn, 
influences  user  perception  about  the  system  (how  efficiently  and  effectively  it  works  toward 
the  user's  goals)  as  well  as  the  organizational  support  behind  the  system  (e.g.,  corrective 
maintenance,  improvement,  supplies,  etc.).  The  user's  perception  of  the  system  and  its 
organizational  support  in  turn  directly  affects  the  user's  acceptance  of  the  system,  in 
addition  to  the  technical  characteristics  of  the  system  and  the  characteristics  of  the  user. 
User  acceptance,  overt  organizational  support,  and  the  user's  personal  stake  in  the  system 
then  determine  how  (or  whether  or  not)  the  system  is  used.  Experience  using  the  system 
then  directly  determines  satisfaction  with  the  system  from  a  user  and  organizational 
standpoint.  Also  generally  believed  to  be  important  factors  (but  not  empirically  confirmed 
with  strong  data  or  consistently  among  researchers)  are:  user  knowledge  of  the  system 
purpose,  user  decision  making  style,  user  job  characteristics,  user/designer  joint  system 
development,  and  user  knowledge  of  the  system  (Lucas,  Ginzberg  and  Schultz,  1990). 
Underlying  the  entire  model  is  the  assumption  th  it  user  acceptance  and  use  are  voluntary; 
the  model  changes  considerably  when  system  use  is  mandatory. 

Under  the  IS  factor  implementation  model,  adoption  and  successful  implementation 
largely  depend  on: 

1)  gaining  support  and  commitment  from  the  user's  management  (e.g.,  funding,  job  re- 
design, organizational  changes,  rewards  and  incentives,  operational  support  and  training) 

2)  seeking  out  potential  users  as  early  adopters  who  have  a  significant  personal  stake  in 
the  problem  the  system  is  designed  to  address,  directly  involving  them  in  the  design  process, 
designing  the  system  to  target  their  technical  needs  as  well  as  personal  characteristics,  and 
focusing  attention  on  their  adoption  and  early  use 

3)  ensuring  that  the  system  addresses  user  personal  stakes  in  system  use. 

Process  Paradigm.  This  paradigm  for    formation  systems  adoption  and 
implementation  research  addresses  the  process  of  organizational  change  and  management 
sup-  ort  behind  system  adoption.  This  paradigm  takes  the  standpoint  that  systems  simply 
ad  .i  ess  organizational  and  user  change  needs  and  provide  a  vehicle  to  implement  those 
ch*  ige  needs.  Therefore,  how  one  implements  a  technological  change  is  the  key  in  this 
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paradigm  to  successful  adoption  and  use.  Three  models  are  prevalent  in  the  IS  adoption 
and  implementation  research  under  the  process  paradigm:  technological  imperative, 
organizational  imperative,  and  emergent  perspective  (Keil,  1991). 

The  technological  imperative  model  is  based  on  the  sociological  assumption  that 
external  forces  (the  environment)  cause  internal  changes,  namely  technological  changes,  to 
user  behavior.  In  consonance  with  innovation  theories,  this  model  revolves  around  two 
change  process  factors:  the  technological  advantage  the  system  provides  a  user  in  performing 
his  or  her  functions,  and  the  system's  ease  of  use.  Together,  these  process  factors  determine 
system  use.  To  promote  adoption,  management  ensures  that  the  system  provides 
technological  advantages  (or  at  least  that  the  benefits  outweigh  the  detriments)  and  that 
the  system  is  technologically  easy  to  use.  Management's  agents  to  this  end  are  IS 
specialists  who  are  trained  in  systems,  the  organization,  how  to  elicit  requirements,  and 
how  to  appropriately  design  systems  for  the  users.  This  model  is  consistent  under  voluntary 
or  mandatory  use  situations. 

The  organizational  imperative  model  assumes  that  people  are  causative  decision 
makers  in  anticipation  or  in  response  to  environmental  changes.  Successful  adoption  and 
implementation  therefore  depend  on  successfully  managing  the  decision  making  and 
implementation  processes.  This  model,  primarily  based  on  the  change  and  innovation  work 
of  Lewin  (1947),  consists  of  three  phases.  According  to  this  model,  successful  change 
depends  on  unfreezing  a  situation  by  creating  a  climate  or  motivation  for  change.  The 
second  phase  consists  of  the  actual  change  based  on  analysis,  design,  development, 
implementation  and  training  for  a  system  and  the  organizational  changes  that  must 
accompany  the  system.  The  final  phase  requires  refreezing  by  institutionalizing  the  new 
system  (wi*h  resulting  organizational  stability).  This  model  (as  shown  in  Figure  2) 
emphasizes  that  an  organization  with  stable  political,  personal,  and  social  coalitions  must 
first  be  disturbed  before  change  can  be  accepted.  Although  there  are  many  roles  (e.g.,  the 
user,  management,  IS  developers),  management  plays  the  key  organizational  rote  in 
directing  the  change  process. 

(Figure  2  found  in  Appendix  F) 

The  key  to  adoption  according  this  model,  therefore,  is  management  awareness  of 
the  need  for  change,  awareness  and  support  for  a  change  vehicle  (the  system  with  attendant 
personnel,  data,  process  and  organizational  structure  changes),  determination  and  follow- 
through  on  changes,  and  institutionalization  of  the  changes.  While  this  model  is  associated 
more  with  mandatory  than  voluntary  IS  adoption,  it  can  apply  equally  to  both  situations. 
Based  on  the  managerial  approach  to  implementing  the  change,  management  can  serve  as  a 
catalyst  to  user  change  as  well  as  an  orchestrator. 

The  final  model  under  the  process  paradigm  is  the  emergent  perspective  model  that 
assumes  people  and  technologies  interact  in  unpredictable  ways.  What's  important  is 
perpetually  adjusting  that  interaction  in  response  to  uncovered  barriers  to  success  (as  shown 
in  Figure  3)  (Leonard-Barton,  1988).  The  key  point  of  this  model  is  that  there  must  be 
mutual  adaptation  between  technological  systems  and  the  organization  (including  the 
organizational  structure,  its  management,  support,  and  the  users).  Change  is  assumed  to 
be  the  norm,  whether  from  internal  or  external  environmental  forces.  No  technological 
system,  the  model  presumes,  can  ever  satisfy  all  organizational  needs  forever  and  will 
therefore  require  continual,  incremental  changes.  Likewise,  no  organization  can  remain 
static  in  light  of  technological  changes  or  opportunities  provided  by  systems. 

(Figure  3  found  in  Appendix  F) 
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The  key  to  adoption  in  this  model  is  the  initial  deployment  of  a  system,  followed  by 
orchestrated  monitoring  and  adaption.  Management  and  users  must  be  willing  to  innovate 
and  to  take  risks  on  the  initial  adoption  and  implementation  with  the  understanding  that 
problems  will  occur.  Management  and  users  must  also  be  willing  to  invest  resources  (e.g., 
time,  personnel,  budgets)  to  identify  and  analyze  implementation  problems.  Most 
importantly,  they  must  be  willing  to  continually  implement  system  and  organizational 
~;ianges  in  a  perpetual  cycle  of  change,  analysis,  and  correction.  In  organizational  terms, 
this  is  conflict  management,  an  essential  feature  of  organizational  management  that  entails 
managerial  processes,  structure,  and  content. 

These  information  systems  adoption  and  implementation  paradigms,  models, 
factors  and  processes  provide  ample  suggestions  for  how  tc  increase  and  improve  successful 
educational  technology  implementation  in  education,  as  well  as  provide  plentiful 
opportunities  for  research. 

Educational  Change  Theory  and  Recommendations  to  Educational  Technology  Adoption  and 
Implementation 

Research  on  educational  change  has  also  produced  knowledge  of  factors  associated 
with  adoption  and  affecting  implementation.  Fullan  (1982)  synthesized  existing  information 
and  reported  the  factors  contained  in  Tables  1  and  2  below. 
FACTORS  ASSOCIATED  WITH  ADOPTION 

1 .  Existence  and  quality  of  innovations 

2 .  Access  to  information 

3.  Advocacy  from  central  administrators 

4.  Teacher  pressure/support 

5.  Consultants  and  change  agents 

6.  Community  pressure/support/apathy/opposition 

7 .  Availability  of  federal  or  other  funds 

8.  New  central  legislation  or  policy  (federal/state/provincial) 

9.  Problem-solving  incentives  for  adoption 

10.  Bureaucratic  incentives  for  adoption. 

Table  1.  Factors  Associated  with  Adoption  (Fullan,  1982,  p.42). 

FACTORS  AFFECTING  IMPLEMENTATION 

A.  Characteristics  of  the  Change 

1 .  Need  and  relevance  of  the  change 

2.  Clarity 

3.  Complexity 

4.  Quality  and  practicality  of  program  (materials,  etc.). 

B,       Characteristics  at  the  School  District  Level 

5.  The  history  of  innovative  attempts 

6.  The  adoption  process 

7.  Central  administrative  support  and  involvement 

8.  Staff  development  (in  service)  and  participation 

9.  Time-line  and  information  system  (evaluation) 

10.  Board  and  community  characteristics. 
C.  Characteristics  at  the  School  Level 

1 1 .  The  principal 

1 2 .  Teacher-teacher  relations 

1 3 .  Teacher  characteristics  and  orientations. 
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D.        Characteristics  External  to  the  Local  System 

1 4 .  Role  of  government 

1 5 .  External  assistance. 

Table  2.  Factors  Affecting  Implementation  (Fullaru  1982.  p.56) 

When  comparing  the  factors  considered  in  educational  change  (above)  to  the  IS 
Implementation  Factor  Model*  a  number  of  apparent  consistencies  can  be  noted.  An 
important  difference  is  that  the  characteristics  described  as  affecting  the  adoption  / 
implementation  process  in  education  arc  stated  and  treated  in  a  static  manner.  What  is 
omitted  is  any  consideration  of  organizational  change.  Even  the  mutual-adaptation 
perspective  which  considers  implementation  as  a  process  in  which  both  the  user  and  the 
innovation  adapts  or  changes,  defines  the  user  narrowly,  and  does  not  consider  changes 
which  may  be  necessary  at  the  organization  or  the  larger  systems  level. 

Educational  technology  adoption  research  and  practice  should  also  bear  in  mind  that 
educational  systems  have  a  characteristic  rarely  seen  in  general  organizations  used  in 
information  systems  research.  Educational  systems  are  professional  bureaucracies  with  a 
unique  organizational  structure,  unique  coordinating  and  controlling  apparati,  user  roles 
and  culture,  communication  channels,  flow  of  decision  making  and  authority,  and  situational 
factors.  For  example,  information  systems  factor  research  consistently  reveals  management 
support  as  the  most  important,  overriding  factor  in  IS  adoption  and  implementation  success, 
but  the  role  of  management  in  a  professional  bureaucracy  is  small,  existing  mainly  to 
provide  resources  to  the  professionals  (i.e.,  educators),  resolve  conflicts  among  the 
professionals,  and  liaise  with  the  external  environment.  In  a  professional  bureaucracy,  a 
successful  decision  to  adopt  an  innovation  won't  likely  be  made  by  the  administration  alone, 
it  will  be  made  and  carried  out  by  individual  professional  educators.  This  characteristic 
does  not  negate  ET  application  of  IS  adoption  models,  however,  it  only  suggests  that  the 
factors  and  processes  for  successful  educational  technology  adoption  will  likely  have  different 
relative  weights  than  the  factors  and  processes  in  previously-researched  successful 
information  systems  adoptions. 

There  also  exists  an  important  difference  between  the  use  of  new  information 
systems  and  the  implementation  of  new  programs  products,  or  technologies  in  education,  at 
the  level  of  the  teacher.  The  way  that  a  teacher  adopts  and  adapts  an  instructional 
innovation  is  affected  by  his  or  her  personal  constructs  concerning  learning  and  instruction 
(Jost,  1992).  In  addition,  classroom  instruction  includes  social  interactions  and 
constructions  which  influence  both  teacher's  thought  processes  and  actions. 

User  acceptance  and  use  has  consistently  been  identified  as  essential  to  informal  .<. 
systems  adoption.  Given  the  professional  bureaucracy  structure  of  the  educational  system, 
professional  educators  rightly  have  the  authority  and  discretion  to  adopt  or  not  adopt 
innovations  for  teaching,  they're  hired  because  of  their  expertise  in  education.  An  important 
factor  to  improve  success  should  be  adopted  from  the  IS  factor  model:  directly  involving  users 
in  the  design  process  and  designing  systems  that  target  their  needs  and  characteristics. 
Without  consideration  of  the  user,  support  and  incentives,  widespread  user  acceptance  by 
existing  educational  professionals  is  unlikely  to  occur.  In  professional  bureaucracies, 
attrition  or  replacement  is  the  most  common  means  of  organizational  sea  changes,  in 
addition  to  changing  the  standards  of  who  can  newly  enter  the  profession,  changing  what 
individuals  learn  in  training  for  the  profession,  and  re-educating  those  professionals  who  are 
willing  to  be  re-educated  (Mintzberg,  1993).  Re-educating  must  take  into  consideration  the 
issues  of  conceptual  change  and  role  changes  as  well  as  technical  and  curricular 
competencies. 


Education  systems  and  educational  change  involve  complex  and  dynamic 
interrelationships.  We  must  expand  our  understanding  of  mutual  adaptation  to  include 
changes  in  the  innovation,  the  teacher,  the  organization  and  the  system.  Successful  change, 
particularly  change  involving  sophisticated  and  pervasive  uses  of  technology,  requires  both 
bottom-up  and  top-down  involvement  and  support. 
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Background 


Information  presented  in  instruction  consists  of  declarative  and  procedural 
knowledge  as  well  as  structural  knowledge  •  the  way  declarative  knowledge  concepts  are 
interrelated  (Jonassen,  1990).  Individuals  selectively  incorporate  this  new  information  into 
their  existing  cognitive  structures  and  then  utilize  the  information  in  these  frameworks 
when  applying  the  knowledge  to  new  situations.   Knowledge  of  content  structure  portrayed 
in  instructional  materials  ('an  help  I  he  leenicr  in  two  ways:  1)  by  organizing  information  in 
short-term  memory  to  luiilil  internal  collections  among  the  new  information  and  2)  by 
integrating  this  information  with  existing  knowledge  structures  in  long-term  memory  to 
develop  internal  connections  (Mayer,  1984), 

Strategies.  In  designing  instruction,  generally  an  expert's  conception  of  the  structure 
of  the  knowledge  is  mapped  onto  the  instructional  materials,  either  implicitly  in  the  text 
structure  and  text  elaboration  techniques  or  explicitly  through  graphic  portrayals  of  the  text 
structure,  Heissner  and  her  colleagues  (1903)  list  types  of  implicit  strategies:  signalling 
(Meyer,  1975),  frames  and  slots  (Armhruster  &  Anderson,  1985),  and  elaboration  theory 
(Rcigcluth  &  Stein,  I9HH),  Kxplicit  strategies  that  graphically  portray  the  structure  of 
knowledge  for  the  learner  or  ask  the  learner  to  construct  such  a  representation  include 
semantic  mapping,  semantic  features  analysis,  structured  overviews,  graphic  organizers, 
spider  maps,  pattern  notes,  concept  maps,  networking,  text  mapping,  schematizing, 
advance  organizers,  and  insertions  such  as  pictures,  questions,  and  verbal  elaboration 
(Heissner,  Jonassen,  &  (Jrahowski,  1993;  Gagne,  1986). 

Mapping  struct  ural  knowledge  onto  instructional  materials  is  thought  to  help 
learners  assimilate  the  new  information  into  their  cognitive  structures  -  adding  to  existing 
schema,  modifying  them,  or  creating  new  ones  as  necessary.  Studies  have  shown  that  a 
student's  cognitive  structure  grows  more  similar  to  that  of  the  instructor  or  the  instructional 
materials  studied  (Shavelson,  1974;  Thro,  1978;  Naveh- Benjamin,  McKeachie  &  Lin,  1989). 

H  ypermedia.  Many  of  these  techniques  assume  the  learner  is  using  a  linear  form  of 
instruction,  similar  to  a  textbook.  The  learner  is  led  through  a  prescribed  path  and  exposed 
to  the  information  in  a  structured  fashion  reflecting  the  expert's  model.  Some  of  the  newer 
technologies,  such  as  hypertext  or  hypermedia,  offer  the  ability  to  give  the  learner  control 
over  his/her  path  through  the  instructional  materials,  hopefully  discovering  the  structure  of 
the  knowledge  in  the  process.  Each  individual  can  take  a  different  path,  encountering 
different  amounts  and  types  of  information  in  different  sequences.     Hypermedia  offers 
both  potentials  and  problems  in  conveying  structural  knowledge  and  assisting  learners  in 
incorporating  this  structural  knowledge  into  their  own  personal  frameworks.  Such  non- 
linear systems  offer  the  ability  to  present  various  complex  structural  representations  in  a 
domain,  to  show  different  points  of  view,  to  show  diversity  among  similar  examples,  and  to 
show  similarities  across  different  categories  (Spiro  and  Jehng,  1990).  They  can  allow  the 
le  1  tier  to  explore  alternatives  and  discover  relationships  (Heller,  1990).  As  the  learner 
maps  out  a  path  through  the  material,  he/she  must  make  choices  and  become  a  more  active 
learner  (Ifourne,  1990). 

At  the  same  time,  such  systems  can  cause  disorientation  in  the  learner,  overtax 
cognitive  processing  capabilities,  encourage  over-simplification,  over-generalizations,  and 
over-compnrtmentalization  (Conklin,  1987;  Hammond;  1989;  Jones,  1987;  Spiro  and  Jehng, 
1990),  Thy  learner  is  required  to  absorb  layers  of  information,  make  it  personally 
meaningful,  and  gain  a  comprehensive  picture  of  the  subject  (Bourne,  1990). 

Hypermedia's  organization  of  nodes  and  links  perpetuates  the  designer's 
assumptions  about  the  nature  of  knowledge  (Doland,  1989),  but  this  structure  is  not  as 
explicit  as  in  traditional  print  materials  (Charney,  1987).  Some  question  whether  this 
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semantic  network  reflected  in  hypertext  can  be  transferred  to  learners  or  whether 
hypertext's  network  is  analogous  to  the  mind's  semantic  network  (Duffy  &  Knuth,  1990; 
Romiszowski,  1990;  Whally,  1990;  Landow,  1990). 

One  of  the  challenges  for  instructional  designers  and  learners  is  how  to  represent 
the  information  and  connections  for  learning  and  information  retrieval.  Research  on  text 
structure  and  comprehension,  schema  development,  metacognition  and  self-regulation, 
learner  control,  and  text  enhancement  techniques  provide  a  basis  for  designing  instruction  to 
improve  acquisition  of  structural  knowledge.  Most  of  this  research  has  dealt  with  linear, 
text-based  instruction.  The  current  study  examined  some  of  these  findings  in  light  of  a  non- 
linear computerized  database  that  allows  the  individual  learner  control  over  his/her  own 
learning.  Horn  (1989)  describes  human  memory  as  associative,  sometimes  connecting 
information  in  unpredictable,  idiosyncratic  ways  and  other  times  in  hierarchical,  structured 
ways.  Jonassen  (1991)  proposes  that  the  node-link  structure  of  hypertext  reflects  the 
semantic  structure  of  the  expert's  knowledge.  Semantic  maps  of  an  expert's  or  experts1 
structure  of  a  subject  could  be  used  to  create  a  node  and  link  structure  in  hypertext. 

Individual  Differences.  Research  has  shown  that  degree  of  prior  knowledge,  i.e., 
existing  schemata,  influences  learning,  reading,  and  inferencing  (Clark,  1990;  Anderson  & 
Pearson,  1984;  Roller,  1990;  Gay,  1986;  Fincher-Kiefer,  1992;  Spilich,  Vesonder,  Chiesi  & 
Voss,  1979;  Chiesi,  Spilich  &  Voss,  1979).  Users  may  have  difficulty  tracking  the  overall 
structure  of  information  in  a  hypertext  system  and  relating  it  to  their  prior  knowledge.  This 
problem  may  be  exacerbated  for  students  with  low  prior  knowledge  by  causing 
disorientation  and  cognitive  overload. 

Verbal  ability  differences  may  also  influence  a  student's  ability  to  process  new 
information  in  a  hypertext  system,  particularly  since  students  with  low  verbal  ability  may 
be  inefficient  in  semantic,  syntactical,  and  pragmatic  processing  of  linguistic  information 
(Hunt,  1983).  Lundow  (1989)  claims  that  some  of  the  new  technologies  change  the  way 
renders  read  because  determining  relationships  among  the  linked  concepts,  which  may  not 
he  obvious  to  the  user,  becomes  central  to  understanding.  This  require*  it  new  type  of 
understanding,  especially  difficult  for  poorer  readers  who  tend  to  strictly  follow  ii  text's 
linear  presentation  and  seldom  use  traditional  connective  aids  such  ns  glossaries  mid 
introductions  in  print  materials.  They  arc  not  used  to  having  to  create  nsHticiiitiotiH  among 
concepts  on  their  own. 

Graphical  Browsers,  New  skills  and  instructional  aids  may  he  required  to  use  these 
technologies.  Explicit  techniques  different  from  those  in  traditional  text  can  help  the  learners 
perceive  the  structure  of  the  knowledge.  Spatial  learning  strategies  provide  a  way  of 
graphically  portraying  structural  knowledge  for  the  learner  or  asking  the  learner  to  construct 
such  a  representation.  Breuker  (1984)  sees  them  as  providing  external  memory  storage  and 
explicitly  depicting  concept  interrelationships.  They  focus  attention  on  relevant  information 
and  help  the  learner  create  internal  and  external  connections  among  the  concepts. 

Maps  cr  graphical  browsers  of  the  structural  knowledge  allow  the  user  of  a  hypertext 
system  to  navigate  from  node  to  node  through  the  structure  while  spatially  portraying  the 
links  between  concepts  (Halasz,  1988).  They  generally  allow  the  user  to  directly  access  any 
node  on  the  map,  speed  up  information  access,  and  prevent  disorientation  (Tsai,  1989). 
"The  learner  can  use  a  semantic  network  as  a  map  of  the  content  knowledge  in  a  curriculum 
and  use  this  map  to  explore  topics  that  are  of  interest  to  him  or  her  at  the  time  the  interest 
is  present"  (Uenenherg,  1988,  p.  309).  Denenberg  recommends  showing  only  the  structure 
m  ound  one  node  to  avoid  overwhelming  the  learner.  This  avoids  some  of  the  problems  of 
Khihnl  maps  (Cunklin,  1987). 

These  gmpliiciil  browsers  are  thought  to  influence  information  processing  by 
liicilitntlliK  both  the  orgiini/ation  and  integration  of  new  information,  similar  to  the  way 
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signalling  techniques  such  as  typographic  cues  and  the  placement  of  certain  words  highlight 
the  text  structure  (Spyriakis,  1989).   Graphical  browsers  draw  the  learner's  attention  to  the 
important  concepts  and  how  they  are  related  to  each  other.  They  can  help  the*  learner  t  ie 
the  new  information  to  prior  knowledge  in  long  term  memory,  especially  when  some  terms  in 
the  graphical  browser  are  already  familiar  to  the  learner.  This  tie  to  already  known  schema 
may  be  most  useful  for  learners  with  high  prior  knowledge  who  recognize  some  concepts  and 
can  use  the  browser  to  help  decide  where  to  go  next.  They  benefit  most  from  this  learner 
control. 

Graphical  browsers  ran  also  help  the  learner  organize  the  new  information  in  short 
term  memory  bv  letting  up  a  framework  to  facilitate  encoding.  This  may  be  more  useful  for 
those  with  low  prior  knowledge  who  have1  little  existing  framework  for  the  new  information 
but  could  potentially  interfere  with  encoding  by  those  with  high  prior  knowledge  who  already 
have  a  well  established  framework.  A  system  without  such  graphical  browsers  would  force 
the  learner  to  generate  his/her  own  framework,  creating  a  more  active  learner.  The 
effectiveness  nf  this  navigation  technique  may  be  tied  to  the  degree  of  prior  knowledge,  both 
of  facts  and  of  the  relationships  between  them,  i.e.,  the  learner's  existing  cognitive 
framework. 

Jonassen  and  Wang  (WW),  1<M)2,  1993)  and  Jonassen  (1991)  have  conducted 
several  studies  on  the  use  of  several  types  of  graphical  browsers  in  hypertext  systems  to 
acquire  structural  knowledge*,  They  found  no  significant  increase  in  structural  knowledge 
and,  in  some  cases,  an  actual  decrease  in  recall.  Only  those  specifically  told  they  would  be 
required  to  create  a  semantic  network  after  the  task  showed  significantly  increased 
structural  knowledge.  Tiioy  concluded  that  merely  showing  the  structural  relationships  was 
not  sufficient  to  result  in  encoding.  In  their  studies  they  used  measurement  techniques  that 
attempted  to  assess  higher  level  skills  and  transfer,  often  achieving  a  basement  effect  in  the 
scores.  Learners  were  unfamiliar  with  these  testing  techniques  and  were  unable  to 
complete  them.  \lso,  these  studies  used  a  very  large  database,  raising  the  question  of  how 
much  information  each  learner  actually  covered. 

Phillips  and  his  colleagues  (1992)  examined  different  types  of  navigational  devices  in 
a  hypertext  database  and  found  that  those  provided  with  the  most  minimal  navigational 
tools  (i.e.,  hotwords  in  the  text  itself)  achieved  the  highest  recall.  This  study  investigated 
only  recall  of  facts  and  concepts,  but  combined  with  the  Jonassen  and  Wang  studies,  it 
raises  the  question  of  how  much  explicit  structure  is  optimal  for  facilitating  information 
processing  by  learners.  The  current  study  examined  both  higher  level  structural  knowledge 
and  declarative  knowledge,  measured  retention  over  time,  measured  structural  knowledge 
using  an  assessment  technique  more  understandable  to  the  learners,  and  compared 
learners  with  different  degrees  of  prior  knowledge  in  the  content  area  and  different  degrees 
of  verbal  ability  to  determine  the  optimal  degree  of  explicit  structure  for  different  learners. 

The  study  examined  the  results  of  using  two  different  graphical  browsers  providing 
different  amounts  of  information  about  the  structure  of  the  knowledge  to  learners  who  have 
been  given  a  specific,  externally  imposed  objective  and  a  structural  knowledge  task  at  the 
beginning  of  instruction.  This  was  compared  to  the  use  of  botwords  embedded  in  the 
instructional  text  itself  showing  no  explicit  structure.  These  pre-lesson  strategies  of 
providing  objectives  and  a  structural  knowledge  task  were  intended  to  alert  the  learners  to 
the  importance  of  acquiring  structural  knowledge  and  to  provide  a  purpose  for  their 
browsing.  Given  that  Jonassen  and  Wang  (1990)  found  many  participants  were  unfamiliar 
with  structural  knowledge  acquisition  and  integration,  hypertext  techniques,  and  methods 
of  assessing  structural  knowledge,  the  study  also  provided  practice  in  basic  hypertext 
navigation  to  reduce  anxiety  and  to  develop  appropriate  strategies  for  navigation  and 
integration  and  familiarity  with  a  graphical  browser  before  the  instruction. 

Several  authors  (Whalley,  1990;  Duchastel,  1990;  Spiro  and  Jehng,  1990)  have 
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suggested  that  hypertext  may  not  be  appropriate  for  many  instructional  uses.  Whalley 
suggests  that  browsing  is  the  most  appropriate  use  of  hypertext.  Browsing  may  not  be  most 
appropriate  for  instructional  purposes,  hut  it  may  he  approprinte  for  facilitating  discovery 
learning  (Bruner,  1960).  Duchastel  (1990)  and  Spiro  and  debug  (1990)  HUggest  that 
hypertext  is  inappropriate  for  highly  structured  learning  tusks.  There  is  little  empirical 
research  to  support  these  claims.  This  study  examined  some  of  them1  criticisms  of  hypertext 
by  using  it  for  just  such  a  highly  structured  content  domain  thut  includes  facts,  concepts, 
and  principles. 

Research  Questions 

These  questions  about  the  educational  use  of  hypertext  led  to  the  following  research 
questions. 

1.  Will  students  given  no  explicit  graphical  representation  of  relationships  among  concepts 
v!  lot  words  group),  those  given  a  graphic  method  for  showing  the  relationships  (Kinks  group), 
mid  those  given  a  gr.'iphie  method  for  allowing  relationships  with  the  relationships  labeled 
(Detailed  Links  groups'*  have  different  levels  of  structural  knowledge  and  perform  differently 
on  tests  of  c  omprehension  and  reeall  of  facts  and  concepts? 

li.  Is  there  an  interaction  between  degree  ol  'prior  hmnrlodfiv  and  type  of  graphical  method  for 
showing  structural  knowledge  no  the  learner's  immediate  and  delayed  structural  knowledge1, 
comprehension,  and  recall  ol  facts  and  concepts? 

U.  Will  the  graphical  method  for  lepresenting  id  met  oral  knowledge  during  instruction  bring 
the  lear  iers'  atiitvtuml  /.on/<7ci//(v  irprvHVHttttiutui  doner  to  that  portrayed  in  the  instructional 
mutvnuln! 

\.  Will  lhi<  type  hI  I'japhical  method  for  representing  structural  knowledge  affect  the 
retention  of  laclH,  concepts,  and  structural  knowledge  over  time! 

.Y  Is  there  on  interact  ion  between  degree  of  verbal  ability  and  type  of  graphical  method  for 
demonstrating  structural  knowledge  on  the  learner's  immediate  and  delayed  structural 
knowledge,  comprehension,  and  recall  of  facts  and  concepts? 

This  study  examined  the  use  of  three  types  of  navigational  techniques  representing 
structural  knowledge  in  e  hypertext  system  for  their  influence  on  the  acquisition  of 
declarative  and  structural  knowledge.   The  effects  of  prior  knowledge,  verbal  ability, 
amount  of  the  content  seen,  and  time  spent  using  the  system  on  achievement  were 
examined.    In  addition,  the  change  in  a  student's  structural  knowledge  and  its  similarity  to 
an  expert  model  were  assessed,  Finally,  retention  over  time  was  assessed  using  the  same 
measures. 

Methods 

Subjects 

One  hundred  thirteen  undergraduate  volunteers  from  a  large  state  university 
received  extra  credit  for  participating  in  both  sessions  of  the  study.  One-hundred  forty- 
started  the  study,  but  five  did  not  have*  KAT  scores,  ten  did  not  return  for  the  second 
session,  and  twelve  did  not  complete  the  first  session.  These  subjects  were  dropped. 


Treatments 

Subjects  were  randomly  assigned  to  a  system  using  hotwords,  one  using  a  graphical 
browser  representing  the  links  between  concepts,  or  one  using  a  graphical  browser  labeling 
the  relationships  between  concepts. 

A  sixty-screen  hypertext  database  on  the  parts  of  the  heart,  circulation,  and  blood 
pressure  based  on  the  materials  of  Dwyer  and  Lamberski  (1977)  was  developed  for  this 
study.  This  content  was  selected  because  it  covers  a  variety  of  facts,  concepts,  and  processes 
and  represents  a  limited  subject  domain  on  which  there  is  general  consensus  on  the 
organization  of  the  content.  An  overarching  structure  for  the  system  with  each  screen 
represented  as  a  node  was  created.  Graphical  browsers  and  hotwords  representing  nil 
nodes  linked  to  each  screen  were  created  for  the  different  treatments.  Links  were 
hierarchical,  heterarchical,  and  following  the  flow  of  blooc  through  the  heart. 

Each  treatment  had  the  same  screens  with  only  the  structural  knowledge 
representation  methods  differing.   The  treatments  were: 

1.  A  group  which  highlighted  the  terms  to  be  linked  that,  when  clicked,  took  the  learner  to  a 
related  screen  without  describing  how  the  two  screens  are  related  (HOTWORDS). 

2.  A  graphical  browser  with  links  visible  but  not  described  but  otherwise  the  same  iik  (he 
third  treatment  (LINKS),  and 

3.  A  graphical  browser  with  links  described  along  the  lines  (DETAILED  LINKS). 

The  learner  could  go  to  each  link  in  succession  from  hotspots  on  the  screen.  By  dirking  on  all 
the  hot  buttons,  the  learner  could  access  the  same  screens  in  all  three  treatment h.  The 
students  were  allowed  to  work  through  the  program  at  their  own  pace,  nccesHing  whatever 
screens  they  saw  tit.  They  could  exit  the  program  at  any  time. 

Instruments 

The  students  were  randomly  assigned  to  the  three  tre  rtment  groups,  based  on  the 
assumption  that  prior  knowledge  and  verbal  ability  scores  would  be  equally  distributed 
among  the  groups. 

The  following  measurements  were  taken  for  each  subject: 

1)  a  pretest  of  prior  physiological  knowledge, 

2)  a  pretreatment  assessment  of  structural  knowledge, 

3)  verbal  Scholastic  Aptitude  Test  score, 

4)  a  posttreatment  survey  of  previous  computer  use  and  attitudes  toward 
computers,  hypertext,  and  this  program, 

5)  the  number  of  different  screens  each  subject  chose  to  view,  and 

6)  the  amount  of  time  spent  on  the  treatment. 

7)  a  40-question  immediate  criterion  posttest  measuring  recall  of  facts  and 
concepts  and  comprehension  consisting  of  three  multiple-choice  testa  and  one 
drawing  test  used  to  assess  recall  and  comprehension  developed  by  Dwyer 
(1978)  (Test), 

8)  an  alternate  form  of  the  40-question  criterion  posttest  administered 
approximately  two  weeks  after  the  immediate  posttest  (Retest),  and 

9)  a  post- treatment  assessment  of  structural  knowledge  (Tree2) 

From  the  Test  and  Retest  scores,  a  change  score  was  calculated  for  the  posttest?   Prom  the 
third,  the  ordered-tree  technique,  a  similarity  measure  to  an  "expert"  structural  knowledge 
representation  and  a  change  in  structural  knowledge  were  calculated. 

The  ordered-tree  technique  has  been  used  in  numerous  studies  to  assess  a  learner's 
cognitive  structure  (NavehBenjamin  and  others,  1986;  McKeithen  and  others,  1981;  Naveh- 
Benjamin,  McKeachie,  and  Lin,  1987;  NavehBenjamin,  McKeachie,  and  Lin,  1989)  and  is 
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based  on  a  theory  of  hierarchical  cognitive  structure  that  assumes  concepts  are  organized 
hierarchically  with  single  concepts  at  the  lowest  level  and  more  abstract  categories  at  the 
higher  levels.  The  technique  uses  recall  theory  which  describes  individuals'  tendency  to  list 
all  of  one  branch  of  the  hierarchy  before  moving  on  to  the  next  branch.  From  a  number  of 
cued  and  uncued  trials,  where  subjects  put  sets  of  low-level  concepts  in  order,  an  algorithm 
finds  the  set  of  all  chunks  for  each  subject  and  creates  representations  of  the  subjects1 
cognitive  structures  (Naveh- Benjamin  &  Lin,  1991).  It  measures  amount  of  organization, 
depth  of  the  hierarchy,  similarity  to  an  "expert"  organization,  and  direction  of  relationships 
between  concepts. 

The  method  was  validated  by  two  content  experts  and  checked  for  reliability  through 
a  field  test.  This  technique  wan  used  to  assess  structual  knowledge  both  before  and  again 
two  weeks  after  the  treatment  in  this  study  to  examine  a  subject's  prior  structural 
knowledge  and  then  the  change  after  the  treatment.  The  pretreatment  and  pesttreatment 
structural  knowledge  tasks  in  this  study  were  scored  using  the  scoring  system  for  t\e 
structural  knowledge  assessment  technique  developed  by  Dr.  Henry  Rueter  of  thr  ognitive 
Science  and  Machine  Intelligence  Laboratory  at  the  University  of  Michigan. 

The  difference  between  the  mean  pretreatment  structural  knowledge  scores  and  the 
mean  retention  posttreatment  structural  knowledge  scores  of  each  group  was  also  used  as  a 
dependent  variable.  The  pretreatment  and  posttreatment  structural  knowledge  scores  for 
each  individual  were  compared  to  the  score  of  an  expert  model  of  structural  knowledge  (as 
represented  in  the  structure  of  the  materials).  A  tree  can  be  very  high  in  structure  and  have 
very  little  similarity  with  the  "expert"  tree.  Such  a  tree  indicates  a  highly  developed, 
individualistic  cognitive  framework  for  this  content.  Another  tree  can  be  highly  similar  to 
the  "expert"  tree  but  only  exhibit  a  moderate  degree  of  structure.  Two  trees  with  similar 
degrees  of  structure  can  look  very  different.  A  change  in  the  degree  of  organization  does  not 
necessarily  indicate  the  resulting  organization  is  closer  to  the  "expert"  one.  Nor  does  a 
consistent  score  from  pretreatment  to  posttreatment  necessarily  mean  there  is  no  change.  It 
is  possible  there  is  a  change  in  the  type  of  organization,  but  the  same  degree  of  organization 
is  evident. 

SAT  verbal  scores  were  requested  to  examine  the  effect  of  verbal  ability  on 
performance  in  the  different  groups.  Prior  knowledge  was  assessed  using  a  general 
physiology  test  developed  by  Dwyer  for  this  purpose.  Although  it  does  not  directly  test 
information  on  the  heart,  performance  on  it  has  been  found  to  correlate  with  the 
performance  on  the  criterion-referenced  tests  (Dwyer,  personal  correspondence). 

A  brief  survey  to  assess  attitudes,  prior  computer  experience,  prior  hypertext 
experience,  and  use  of  the  hypertext  lesson  '/as  used  immediately  after  the  treatment. 
Followup  interviews  with  a  sample  of  learners  were  conducted  by  trained  interviewers  to 
provide  additional  qualitative  data. 

The  audit  trails  for  each  student  were  kept  by  the  computer  program.  This  included 
paths  taken  and  time  spent  at  each  point  in  the  program.  A  count  of  the  number  of 
different  screens  seen  was  displayed  on  each  screen  and  recorded  by  the  computer.  The 
total  time  spent  on  the  lesson  was  also  recorded. 

Procedure 

Verbal  SAT  scores  were  obtained  as  a  measurement  of  verbal  ability.   Before  the 
treatment  subjects  took  a  basic  physiology  prior  knowledge  test  developed  by  Dwyer,  used 
the  ordered-tree  technique  of  Reitman  and  Rueter  (1980)  to  measure  their  existing 
structural  knowledge  of  the  concepts  in  the  lesson,  and  practiced  using  the  navigational 
technique  in  the  treatment. 

For  up  to  an  hour,  the  subjects  were  provided  with  four  objectives  for  the  lesson  and 
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then  navigated  through  as  many  of  the  sixty  screens  in  the  system  as  they  chose  to  using 
on?  of  the  three  randomly  assigned  navigational  techniques,  and  then  completed  an 
experience  and  attitude  survey  and  a  posttest  measuring  the  ability  to  construct  and  label  a 
diagram  of  the  heart,  knowledge  of  facts,  identification  of  concepts,  and  comprehension.  A 
sampling  of  the  students  were  interviewed  after  the  treatment.  Path  data  was  recorded  by 
tlii»  computer  for  each  participant.  Approximately  two  weeks  later  the  students  took  a 
delayed  retention  posttest  and  completed  a  second  ordered-tree  to  assess  structural 
knowledge. 


Data  Analysis 

Regression  was  used  to  test  for  the  effect  of  the  treatments,  the  influence  of  verbal 
ability  and  prior  knowledge  on  recall  and  retention  of  facts  and  concepts,  comprehension, 
and  structural  knowledge,  and  the  interaction  between  variables  and  to  find  the  most  valid 
predictor  variables;  those  which  account  for  the  most  variance  in  the  dependent  variables. 
This  was  chosen  over  a  multivariate  analysis  of  variance  to  preserve  the  maximum  amount 
of  data  on  each  independent  variable.  Multiple  regression  was  used  to  determine  the 
function  that  best  predicts  performance  on  the  each  of  the  dependent  variables:  immediate 
posttest,  difference  between  delayed  and  immediate  posttests,  posttreatment  structural 
knowledge  task,  similarity  of  a  subject's  structural  knowledge  to  an  "expert11  representation, 
difference  between  pre  and  posttreatment  structural  knowledge  tasks  and  to  test  the  effect 
of  the  treatments  on  each  of  these  variables. 

The  variables  representing  coverage  of  the  hypertext  database  (Count)  and 
persistence  (Etime)  were  also  included  as  predictors.  In  examining  the  raw  data,  these 
variables  differed  widely  among  the  subjects,  so  their  influence  on  the  dependent  variables 
were  also  sought  to  reduce  their  potential  confounding  effects. 

For  each  dependent  variable,  the  hypothesized  predictors,  the  potential  confounding 
variables,  and  the  interaction  of  all  these  variables  and  the  treatment  were  regressed. 
First,  parameter  estimates  were  obtained  for  all  the  predictor  variables  and  the  interactions 
of  these  variables  with  the  treatment.  Those  that  were  not  significant  at  the  ,05  level  were 
dropped  and  the  equation  refit.  The  resulting  function  was  the  fit  fur  that  dependent 
variable  from  the  predictor  variable**  given,  indicating  which  variables  were  significant 
predictors  for  each  dependent  variable  and  what  percentage  of  the  total  variance  of  the 
dependent  variable  they  accounted  for  as  a  group  (i.e.,  the  multiple  correlation  coefficient). 
When  interactions  between  any  of  the  predictors  and  the  treatment  were  found,  equations 
were  fit  for  each  treatment. 

A  method  developed  by  the  researcher  was  used  to  compare  the  paths  the  students 
used.  This  technique  was  carried  out  for  a  sampling  of  twelve  subjects  from  each  treatment. 
It  reported  information  on  how  often  they  went  forward  or  backward,  where  they  clicked, 
and  whether  they  moved  hierarchically,  heterarchically,  or  following  the  blood  flow. 

The  survey  and  interview  data  were  summarized  quantitatively  and  qualitatively. 
Data  on  prior  computer  and  hypertext  use  and  perceptions  were  correlated  with  the  test  and 
structural  knowledge  results. 

Results 

The  three  treatment  groups  showed  no  significant  differences  on  verbal  Scholastic 
Aptitutde  Test  scores  or  prior  knowledge  of  physiology.  Groups  means  showed  a  difference 
between  the  immediate  posttest,  change  from  immediate  to  delayed  posttest,  posttreatment 
ordered-tree  task,  and  posttreatment  tree  similarity  to  an  "expert"  tree  among  the  groups. 


796 


Subjects  in  the  Hotwords  treatment  group  consistently  scored  the  lowest  except  in  the 
difference  between  the  pre-  and  posttreatment  trees  with  those  in  the  Links  and  Detailed 
Links  groups  being  more  similar  to  each  other.  The  Detailed  Links  group  generally  had  the 
smallest  within  group  variance  except  on  the  difference  between  the  immediate  and  delayed 
posttests  and  the  difference  between  the  pre*  and  posttreatment  trees  (see  Tables  1,2,  and 
3).  On  this  latter  variable  there  is  a  marked  difference  in  the  Detailed  Links  group.  They 
had  a  very  small  difference  between  the  pre-  and  posttreatment  scores  (i.e.,  the  post  scores 
were  only  slightly  higher). 

Table  1 

Means  and  Standard  Deviations  on  Immediate  Posttest,  Delayed  Posttest,  and  Change  in 
Posttest  Scores  by  Treatment  Group 


Treatment 
Group 


Immediate 
Posttest 


Delayed 
Posttest 


Change  from  Immediate 
to  Delayed  Posttest 


M  SD 
20.919  (8.207) 
23.237  (K.221) 


M  SD 
17.838  (7.984) 
19.895  (7.266) 
19.132  (6.757) 
18  965  (7.043) 


M  SD 
3.081  (4.159) 
3.342  (5.800) 
4.526  (5.331) 
3.655  (5.144) 


Hotwords 
(n=37) 
Links 
(n=38) 

Detailed  Links23.65H  (7,549) 
(11-38) 

All  Groups       22.619  (8,01(0 
(n-113) 

Note:  The  chxmK"  in  posttont  wort's  was  computed  by  subtracting  the  immediate  posttest 
score  from  the  delayed  posttoHt  score.  A  positive  number  indicates  a  drop  in  the  score  over 
time. 
Table  2 

Means  and  Standard  Deviations  on  Posttreatment  Ordered-Tree  Task,  Change  in 
Structural  Knowledge,  and  Similarity  to  an  "Expert"  Ordered-Tree  by  Treatment  Group 


Treatment 
Group 


Ordered-Tree 
Task 


Change  from 
Pre  to  Post 
Ordered-Tree 


Similarity 
to  "Expert" 
Ordered-Tree 


M  SD 
16.243  (13.621) 
20.158  (14.045) 
19.579  (11.758) 


M  SD 
2.243  (14.504) 
3.368  (15.958) 
.058  (11.832) 


M  SD 
22.162  (20.581) 
33.289  (22.445) 
27.447  (20.201) 


Hotwords 
(n=37) 
Links 
(n=38) 
Detailed 
Links 
(n=38) 
All  Groups 
(n=113) 

Note:  The  change  in  ordered -tree  scores  was  computed  by  subtracting  the  pretreatment 
ordered  tree  score  from  the  posttreatment  score.  A  positive  number  indicates  an  increase  in 
the  amount  of  structure. 


18.947  (13.315)      1.885  (14.139) 


27.681  (21.404) 
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Table  3 

Means  and  Standard  Deviations  on  Number  of  Screens  Seen  and  Time  in  Treatment  by 
Treatment  Group 

Treatment  Group  Screen  Count  Elapsed  Time  (in 

seconds) 


M  SD 

M  SD 

Hotwords 

43.9  (12.7) 

1113  (609) 

(n=37) 

Links 

45.1  (10.5) 

1293  (636) 

(n=38) 

Detailed 

44.9  (12.7) 

1360  (564) 

Links 

(n=38) 

All  Groups 

44.7  (11.0) 

1257  (607) 

(n=113) 

While  not  included  in  the  hypotheses,  both  time  on  treatment  (Etime)  and  number 
of  different  screens  seen  (Count)  were  included  in  the  analysis  since  Elapsed  Time 
represents  the  degree  of  persistence  or  use  of  the  navigational  tools,  depending  on  how  many 
total  screens  were  seen,  and  Screen  Count  represents  the  actual  coverage  of  the  content. 
The  latter  indicates  the  number  of  different  screens  seen  out  of  the  sixty  possible.  Although 
they  are  highly  correlated  (r=  61378),  both  were  included. 

Immediate  Posttest 

Multiple  regression  analysis  revealed  predictors  of  performance  on  the  immediate 
posttest  were  treatment  group,  prior  knowledge,  an  interaction  of  number  of  screens  seen 
and  treatment  group,  and  an  interaction  of  time  spent  on  the  treatment  and  treatment. 
Approximately  sixty  percent  of  the  total  variance  in  the  criterion  variable  Test  can  be 
accounted  for  by  these  predictors  (R-square=. 604873,  F=17.52,  £=0001),  The  predictors  of 
this  variable  were  then  tested  for  their  significance  as  main  effects  and  in  interaction  with 
treatment  adjusted  for  all  other  variables  in  the  regression  (see  Table  4). 

A  significant  main  effect  was  found  for  Prior  Knowledge,  Number  of  Screens  Seen, 
Elapsed  Time,  and  Treatment.  Because  there  was  a  significant  interaction  between  two  of 
the  predictor  variables  and  the  treatment,  equations  were  determined  by  treatment.  The 
equation  for  each  treatment  is  listed  in  Table  5. 

Table  4 

Test  of  Significance  of  Predictor  Variables  for  the  Immediate  Poatteat  Criterion  Variable 


Source 

df 

Sums  of  Squares 

Mean  Square 

F 

P 

Prior 

1 

1567.6573 

1567.6573 

56.78 

.0001* 

Count 

1 

159.0293 

159.0293 

5.76 

.0182* 

Etime 

1 

276.0847 

276.0847 

10.00 

.0021* 

Treatment 

2 

190.4967 

95.2484 

3.46 

.0355* 

Count*Treat 

2 

376.6657 

188.3328 

6.82 

.0017* 

Etime*Treat 

2 

248.8115 

124.4058 

4.61 

.0133* 
*B<.05 
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The  same  predictor  variables  were  not  significant  in  the  equations  for  each  of  the 
treatments.  Prior  Knowledge  remained  significant  in  a  positive  direction  in  all  three 
treatment  equations.  Larger  beta  weights  for  the  same  variable,  regardless  of  sign, 
contribute  more  to  the  prediction  (Huck,  C  n-mier,  &  Bounds,  1974,  pp.  158)>  although  them* 
numbers  should  not  be  used  for  comparison  when  the  predictors  are  correlated  and  not 
orthogonal  as  they  are  in  this  Htudy  (Darlington,  1968).  Beta  weights  cannot  be  compared 
across  variables  due  to  different  mcaHurement  scales.  For  variables  such  as  Elapsed  Timo. 
the  beta  weights  were  relatively  biiuiII  due  to  the  scale  being  used  (i.e.,  seconds). 

Table  5 

Functions  for  Immmliatu  Puattiml  fur  Kach  Treatment 


Equation  H-iiqiiiirt*  Variable  Parameter  p 


Treatment  1 

tSHfirt 

Intercept 

-li. 7907 77 

.0326* 

(Hotwords) 

.000) « 

Prior 

1.385386 

Count 

-  .178874 

.0636 

Etime 

.008288 

.0001* 

Treatment  2 

.11.140 

Intercept 

-13.466263 

.0198* 

(Links) 

Prior 

.768232 

.0018* 

Count 

.340977 

.0009* 

Etime 

.002544 

.1210 

Treatment  3 

.5164 

Intercept 

-  9.501131 

.1094 

(Detailed 

Prior 

.962467 

.0005* 

Links) 

Count 

.236039 

.0275* 

Etime 

.000085 

.9689 

£<.05 

A  plot  of  the  predictor  variable  Prior  Knowledge  against  the  predicted  immediate 
posttest  score  showed  the  positive  direction  of  its  influence.  It  was  a  significant  predictor  in 
all  three  groups  (*p<.05),  but  there  was  no  significant  interaction  between  prior  knowledge 
and  the  treatment.  The  steepest  slope,  and  hence,  the  strongest  influence  was  evident  in 
the  Hotwords  group,  but  this  difference  was  not  statistically  significant.  The  Links  group 
had  the  lowest  parameter  estimate  and  the  Ibast  influence  from  prior  knowledge. 

The  second  significant  main  effect  was  the  predictor  Count  which  indicated  the 
number  out  of  the  total  sixty  screens  each  subject  looked  at  in  the  lesson.  There  was  a 
significant  interaction  between  treatment  and  the  number  of  screens  seen.  The  Links  and 
Detailed  Links  group  showed  a  positive  relationship  but  the  Hotwords  group  showed  a 
small,  slightly  negative  relationship.  There  was  very  little  difference  in  the  parameter 
estimates  for  the  Links  and  Detailed  Links  groups.   The  more  screens  subjects  in  both  of 
these  groups  tended  to  view,  the  higher  their  immediate  posttest  score,  whereas,  for  the 
Hotwords  group,  the  rumoer  of  screens  viewed  did  not  make  a  difference. 

The  Elapsed  Time  predictor  indicated  how  long  each  subject  spent  on  the  treatment, 
which  may  be  interpreted  as  a  measure  of  persistence.  Only  in  the  Hotwords  group  was  the 
predictor  variable  Elapsed  Time  significant  at  the  .05  level.  Some  of  this  may  have  been 
due  to  collinearity  of  the  predictor  variable,  in  particular  the  correlation  between  Screen 
Count  and  Elapsed  Time.   An  outlier  who  viewed  the  lesson  for  almost  3500  seconds  may 
have  produced  the  significant  effect  from  Elapsed  Time  in  this  treatment  where  it  is  not 
significant  in  the  others. 
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This  analysis  for  the  immediate  posttest  indicated  a  significant  main  effect  for 
treatment  at  the  .0355  level  with  the  Links  and  Detailed  Links  scores  higher  than  the 
Hotwords  group.  Trentnient  interacted  significantly  with  the  number  of  screens  viewed  and 
the  time  spent  on  the  treatment.  Prior  knowledge  was  a  significant  predictor  of  immediate 
posttest  scores  in  all  three  treatments,  but  there  was  no  significant  interaction  between 
treatment  group  and  prior  knowledge.  Students  with  differing  degrees  of  prior  knowledge 
did  not  perform  signifiemitly  different  in  the  three  treatments.  The  predictor  Verbal  SAT 
scored  used  to  represent  verbal  ability  did  not  contribute  to  prediction  of  the  criterion 
Immediate  Posttest. 

Change  from  Pretest  to  Posttest 

/ 

This  variable  was  formed  by  subtracting  the  delayed  posttest  score  from  the 
immediate  posttest  score.  A  positive  number  indicates  the  loss  in  score.  A  negative  number 
indicates  a  higher  score1  on  the  delayed  posttest.  Predictors  of  change  in  score  from 
immediate  to  delayed  posttest  were  prior  knowledge,  immediate  posttest  score,  treatment, 
and  an  interaction  between  verbal  ability  and  treatment  (R-square=  386828,  F=9.46, 
11=0001). 

The  variables  Prior  Knowledge,  Immediate  Posttest,  and  Treatment  were 
significant  (see  Table  6).  On  this  criterion  variable,  the  subjects  in  the  Hotwords  group 
(M=3.081)  had  signficantly  less  loss  in  achievement  between  the  immediate  and  delayed 
posttest  followed  by  the  Links  group  (M=3.342)  and  then  the  Detailed  Links  group 
(M=4.526).  The  work  of  Phillips  and  others  (1992)  showed  that  the  subjects  given  the  least 
amount  of  structure  performed  better  on  recall.  In  the  current  study,  those  given  the  least 
structure  performed  the  worst,  but  they  lost  less  over  time.  The  pattern  of  loss  supports  the 
statistical  concept  of  regression  to  the  mean  over  time. 

Table  6 

Test  of  Significance  of  Predictor  Variables  for  the  Difference  between  Immediate  and 
Delayed  Posttests  Criterion  Variable 


Source 

df 

Sum  of 

Mean 

F 

P 

Squares 

Square 

Prior 

1 

239.49088 

239.49088 

13.84 

.0001* 

Trent 

2 

131.11542 

65.55771 

3.79 

.0258* 

Test 

1 

749.61628 

749.61628 

43.31 

•O001* 

Verbal 

1 

57.14237 

57.14237 

3.30 

.0721 

Verbal*Treat 

2 

136.95793 

68.47896 

3.96 

.0220* 

ft<.05 

Because  there  was  a  significant  interaction  between  one  of  the  predictor  variables,  Verbal 
SAT  score,  and  the  treatment,  equations  were  determined  for  each  treatment  (see  Table  7). 
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Table  7 

Functions  for  the  Difference  between  Immediate  and  Delayed  Pnsttests  for  Each  Treatment 


Equation 

R-square 

Variable 

Parameter 

p 

Treatment  1 

.2594 

Intercept 

11.291464 

.0230* 

(Hot  words) 

Prior 

-  .0547680 

.0207* 

Test 

.342397 

.0022* 

Verbal 

•  .006322 

.4070 

Treatment  2 

.3895 

Intercept 

•4.437707 

.4290 

(Links) 

Prior 

•  .480898 

.0501 

Test 

.412299 

.0023* 

Verbal 

.020062 

.0539 

Treatment  3 

.4899 

Intercept 

•3.736668 

,4115 

(Detailed 

Prior 

•  .443104 

.0600 

Links) 

Test 

.550589 

.0001* 

Verbal 

.012112 

.1572 

£<,05 

Thirteen  students  actually  improved  between  the  immediate  and  delayed 
posttests  (i.e.,  two  in  the  Hotwords  group,  eight  in  the  Links  group,  and  three  in  the 
Detailed  Links  group).  The  predictor  Prior  Knowledge  contributed  til  prediction  of  tin* 
change  between  Immediate  and  Delayed  Posttests.  A  plot  of  the  predicted  difference 
between  immediate  and  delayed  posttest  scores  for  each  of  the  group*  allowed  u  Hlight 
negative  influence  of  prior  knowledge,  although  the  predictor  wan  Kighifirant  itt  the  .05  level 
only  for  the  Hotwords  group  (jp. 0207).  It  approached  Higtiifu  iu.ee  in  the  Link*  group 
(£=.0501)  and  in  the  Detailed  Links  group  (fl=  OtfOO).  There  wan  no  Higtilfieiint  interaction 
between  treatment  group  and  degree  of  prior  knowledge.  TIiIh  variable  roprcHcntH  retention 
of  knowledge  from  immediately  after  the  treatment  to  t  wo  week*.  A  negative  parameter 
estimate  here  indicates  that  those  with  low  prior  knowledge  had  ft  higher  difference  between 
immediate  and  delayed  posttest  scores.  The  different  treatments  did  nut  help  or  hinder 
those  with  low  prior  knowledge. 

Score  on  the  immediate  posttest  was  a  significant  predictor  of  the  difference  between 
the  immediate  and  delayed  posttest  scores  in  all  groups.  Those  who  had  higher  immediate 
posttest  scores  tended  to  decrease  more  between  immediate  and  delayed  posttest  than 
those  who  had  lower  immediate  posttest  scores.  The  different  treatments  aid  not  interact 
with  immediate  posttest  score  to  influence  retention,  The  influence  was  strongest  in  the 
Detailed  Links  group  followed  by  the  Links  group  and  then  the  Hotwords  group. 

Since  verbal  ability  was  not  significant  as  a  main  effect  but  interacted  with 
treatment,  plots  of  this  variable  and  the  predicted  value  of  the  criterion  variable  were  drawn 
for  each  treatment.  Only  in  the  Links  group  did  verbal  ability  approach  significance  as  a 
predictor  (p=,0539).  It  was  not  a  significant  predictor  in  the  other  groups.  This  may  have 
been  due  in  part  to  its  correlation  with  prior  knowledge  which  was  a  significant  predictor 
overall,  but  did  not  interact  with  treatment 

Posttreatment  Structural  Knowledge 

Predictors  of  posttreatment  Htructural  knowledge  were  pretreatment  structural 
knowledge,  an  interaction  (if  prior  knowledge  and  treatment,  and  an  interaction  of 


immediate  posttest  score  and  treatment.  The  multiple  correlation  coefficient  predicted 
about  thrity-four  percent  of  the  total  variance  (R-square=  335604,  F=5.78,  £=.0001). 

There  was  a  significant  main  effect  from  the  pretreatment  ordered-tree  task.  The 
pretreatment  score  may  reflect  both  a  degree  of  prior  knowledge  about  the  organization  of 
the  concepts  and  a  comfort-level  with  the  task  itself.  These  task  scores  represent  the  decree 
of  structure,  not  the  structure's  similarity  to  any  stnndard.  A  student  may  have  n  high 
degree  of  structure  on  both  the  pre-  and  posttreatment  tasks,  but  the  results  may  represent 
two  entirely  different  structures.  Because  there  was  a  significant  interaction  between  two  of 
the  predictor  variables,  Prior  Knowledge  and  Immediate  Posttest,  and  the  treatment, 
equations  were  determined  for  each  treatment  (see  Table  B). 
Table  8 

Test  of  Significance  of  Predictor  Variables  for  the  PqHttrualm.mit  Ordered- Tree  Task 
Criterion  Variable 


Source  df 


Sum  of 


Treat  2 

.0767 
Treel  1 

.0003* 
Prior  1 
Prior*Treat  2 

.0431* 
Test  1 
Test*Treat  2 

.0052* 


Squares 

659.9262 

1793.5448 

298.4925 
812.3316 

157.3070 
1388.3069 


Mean 
Square 

329.9631 

1793.5448 

298.4925 
406.1638 

157.3070 
694.1535 


V 

2.63 

14.31 

2.38 
3.24 

1.26 
5.54 


.1258 


.2651 


*fl<.05 
Table  9 

Functions  for  the  Posttreatment  Ordered-Tree  Task  for  Each  Treatment 


Equation 

Treatment  1 
(Hotwords) 


Treatment  2 

(Links) 

.113976 


Treatment  3 

(Detailed 

Links) 


R-square 
.3005 

.3897 
.4983 

.3477 


Variable 

Intercept 
Treel 
Prior 
Test 

Intercept 


Prior 
Test 

Intercept 
Treel 
Prior 
Test 


Parameter 

-23.910117 
.300901 

1.893467 
-  .329488 

3.590528 


Treel 


-  .501087 
1.136821 

13.722323 
.510772 

-  .088868 

-  .086433 


.0688 

.0939 

.0125* 

.3249 

.7643 


.4137 

.0009* 

.1838 

.0002* 

.8707 

.7564 


*B<.05 
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There  was  a  definite  linear  relationship  between  pre-  unci  pciHtt rc*at  itUMit  scores  with 
those  performing  higher  before  the  treatment  also  performing  higher  after  the*  tt  iMit turnt, 
the  plot  of  pretreatment  ordered-tree  task  by  predicted  posttroattiient  ordered  liw  tnnk  tliil 
not  seem  to  indicate  that  a  demonstrated  understanding  mid  ability  tci  do  the  tank  htiforn 
the  treatment  resulted  in  much  better  posttreatment  structure.  Tin1  plot  seemed  to  indicate1 
that  the  overall  trend  was  to  perform  slightly  worse  on  the  posttreatment,  tank,  especially  at 
the  higher  range  of  the  pretreatment  scores.  Although  there  was  not  a  significant 
interaction  between  pretreatment  and  posttreatment  ordered tree  seorea,  only  in  the  function 
for  the  Detailed -Links  group  was  pretreatment  scon*  a  significant  predictor  of  posttreatment 
score. 

The  immediate  posttest  interacted  with  the  treatment  but  wan  not  a  predictor  of 
posttreatment  ordered-tree  score  by  itself.  It  was  a  significant  predictor  only  in  the  Links 
group.  In  the  Links  group  there  was  a  definite  linear  pattern  showing  a  pnHitive 
relationship  between  immediate  posttest  score  and  predicted  wore  mi  the  posttreatment 
ordered-tree  task.  There  was  no  obvious  explanation  for  thin  one  group  performing  so 
differently,  especially  since  the  Detailed  Links  group  often  behaved  like  the  Links  group  and 
tended  to  demonstrate  similar  patterns.   It  was  expected  that  the  immediate  posttest 
would  have  some  predictive  value  for  the  structural  knowledge  taHk  as  structural  knowledge 
is  thought  to  be  necessary  to  answer  comprehension  quest  ions.  Previous  studies  have  not 
shown  it  to  be  necessary  for  factual  recall  and,  in  some  (rases,  have  found  it  to  be  a 
hindrance  (Jonassen  &  Wang,  1992). 

Prior  knowledge  interacted  with  treatment  but  wus  not  a  predictor  of  posttreatment 
ordered-tree  score  by  itself.  It  was  a  significant  predictor  only  in  the  Hotwords  group.  The 
plot  showed  a  definite  positive  relationship  with  higher  prior  knowledge  students  doing 
better  on  the  posttreatment  ordered-tree  task. 


Change  in  Structural  Knowledge 

This  variable  represents  the  change  in  structural  knowledge  from  before  the 
treatment  to  two  weeks  after  the  treatment.  A  positive  number  indicates  an  increase  in  the 
amount  of  structure  while  a  negative  number  indicates  a  decrease  in  the  amount  of 
structure.   Predictors  of  change  in  structural  knowledge  were  prior  knowledge  and 
pretreatment  structural  knowledge  (R-square=.321461,  F=26.06,  ft=.0001).   There  was  no 
significant  interaction  between  prior  knowledge  and  the  treatment,  but  there  was  a  positive 
relationship  between  prior  knowledge  and  change  in  structural  knowledge  (jp.Olfll).  Those 
with  higher  prior  knowledge  had  a  significantly  greater  increase  in  structural  knowledge 
than  thoBe  with  lower  prior  knowledge.  It  is  important  to  remember  the  previously  detailed 
dilficultics  with  the  posttreatment  structural  knowledge  task  when  interpreting  these 
results. 


Table  10 

Test  of  Significance  of  Predictor  Variables  for  the  Dif  ference  between  the  Pretreatment  and 
Posttreatment  Ordered-Tree  Tasks 

Source  df        Sum  of  Mean  V  p 

Squares  Square 

Prior  1  943.3673  943.3673  (>.H3  .0 

Two  I  1       6820.5012        6820.5012  49.38  .0< 

*fi<.05 
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A  plot  of  the  predicted  difference  between  pretreatment  and  posttreatment  ordered- 
tree  task  by  pretreatment  ordered-tree  task  showed  a  negative  relationship  between  these 
variables.  This  can  be  interpreted  to  mean  that  the  better  one  did  on  the  initial  structural 
knowledge  task,  the  less  difference  there  was  between  the  pre  and  post  scores  on  the  task 
to  a  point.  In  all  the  groups,  there  was  a  point  where  the  subjects  did  better  on  the 
pretreatment  task  than  on  the  posttreatment  task.  About  forty-three  subjects  fall  in  this 
area.  This  loss  of  structure  was  most  evident  in  those  with  higher  scores  on  the 
pretreatment  task. 

Structural  Knowledge  Compared  to  an  "Expert" 

The  prettenl  meat  and  poHttreatment  ordered-trees  of  each  subject  were  compared  to 
each  oilier  ami  to  the  ordered-tree  representing  the  structure  of  the  information  in  the  lesson 
um  created  by  I  lie  instructional  developer  (i.e.,  the  "expert"  ordered-tree)  using  the  program 
I  ITHhii  I  0  developed  hv  Dr.  Henry  Hueter  at  the  Cognitive  Science  and  Machine  Intelligence 
Laboratory  at  the  I  (militate  School  of  Husiness  at  the  University  of  Michigan.  This 
Icchnlijuc  computes  the  llirtle  similarity  measure  between  ordered  trees.  This  is  the  ratio 
nf  I  ln>  rh  11  ok  h  in  common  ami  the  total  chunks  of  two  ordered  trees.  A  similarity  measure  of 
I  0  imlicates  two  identical  trees  and  0  indicates  two  trees  that  are  maximally  dissimilar. 
The  original  similarity  scores  were  multiplied  by  ten  and,  therefore,  ranged  from  100  for 
identical  trees  to  zero  indicating  total  lack  of  similarity  between  two  trees.  Differences  in 
directionality  between  two  trees  do  not  enter  into  the  measure,  so  this  measures  loses  some 
af  the  information  in  each  tree,  but  it  is  the  best  available  at  this  time.  Average  similarity 
measures  are  shown  in  Table  11. 

Table  11 

Mean  Similarity  Measures  for  Pretreatment  and  Posttreatment  Ordered  Trees  Compared  to 
an  "Expert"  Tree 

Overall  Hotwords         Links  Detailed 

Links 


MSJ2  M      SI)       M      SB  MSE 

Pretreatment              29.46  (20.09)  27.24  (20.28)      31.76  (19.55)  29.32  (20.56) 
/Kxpert 

Posttreatment             27.68  (21.40)  22.16  (20.58)      33.29  (22.45)  27.45  (20.20) 
/Kxpert 

Difference  between       •  1 .7H  (20,8(1)  .5,08(19.68)        1.53  (26,30)  -1.87  (15.06) 
Pre  &  Posttreatment 
Similarity  Scores 

Pre/Poattreatment.       35.74  (28.57)  31.43(28.30)      37.61  (27.72)  38.08  (29.92) 
Similarity  (within  subject) 

Note:  Pretreatment/Kxpert  is  the  similarity  between  a  subject's  pretreatment  ordered-tree 
score  and  that  of  the  "expert."  I'oHttrcatnieiit/Kxport  in  the  similarity  between  a  subject's 
posttreatment  ordered-tree  m  are  and  that  of  the  "expert."  Pretrentment/Posttreatment 
Similarity  is  the  similarity  measure  between  a  HiihjectV  pre  and  posttreatment  trees, 
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Difference  between  Pre  &  Posttreatment  Similarity  Scores  is  found  by  subtracting  the 
pretreatment  ordered-tree  score  from  the  posttreatment  score. 


The  results  of  this  analysis  must  be  interpreted  cautiously  as  the  ordered- 
tree  technique  proved  difficult  and/or  frustrating  for  many  students.  Kight  students 
showed  no  organization  at  all  in  the  pretreatment  task  which  may  have  boon  due  to 
a  lack  of  knowledge  or  difficulty  with  the  task.  The  overall  moan  structure*  scon? 
was  only  17  before  the  treatment  and  19  after  the  treatment.  ThiH  could  indicate* 
that  the  task  did  not  discriminate  well  between  students  with  and  without 
structural  knowledge. 

Although  the  statistical  analysis  indicated  no  significant  difference  among 
the  treatment  groups  on  the  posttreatment-Mexpert"  similarity  scores  (K=Z.H1, 
£=.0781),  there  was  a  marked  difference  between  the  Hotwords  group  and  the  Links 
group  with  the  Hotwords  group  much  lower  in  similarity  to  the  structure  represented 
in  the  lesson  (see  Tablell).  The  Hotwords  group  also  demonstrated  lower  similarity 
in  their  pretreatment  trees  and  posttreatment  trees  (M=31.43).  On  the  average, 
their  posttreatment  trees  were  over  five  points  lower  than  their  pretreatment  trees. 

The  only  significant  predictor  of  degree  of  similarity  to  the  "expert"  structure 
of  knowledge  presented  through  the  links  in  the  hypertext  system  was  an  interaction 
of  immediate  posttest  and  treatment.   There  was  no  significant  main  effect.  The 
multiple  correlation  coefficient  between  these  predictors  and  the  criterion  was 
.219811  (F=3.47,  £=.0014,  see  Table  12). 

Table  12 

Test  of  Significance  of  Predictor  Variables  for  the  Comparison  of  Posttreatment 

Qrdered-Tree  Taak  and  the  "Expert"  Qrdered-Tree 

Source  df  Sum  of  Mean  F  p 

Squares  Square 

Treatment  2  1315.6904        657.8452  1.69 

Pnor*Treatment  3  2670.9577  890.3192  2.29 

Tt'Ht*Trentment  3  5377.1967        1792.3989  4.60 

*fi<.05 

A  positive  relationship  between  the  predictor,  Test,  and  criterion  variable 
was  indicated  in  both  the  Links  and  Detailed  Links  groups,  although  it  was  only 
significant  in  the  Links  group  (£=.0014). 

The  interaction  of  prior  knowledge  and  treatment  approached  significance  , 
so  further  uualysis  was  performed.  In  the  Hotwords  group,  prior  knowledge 
approached  significance  (p=.0562)  which  would  seem  to  follow  the  trend  for  many  of 
the  dependent  variables  of  prior  knowledge  being  most  influential  in  the  Hotwords 
group.  It  exerted  a  strong  positive  influence  on  the  posttreatment  similarity  score  in 
this  group.  This  also  seemed  to  be  indicated  by  the  plot  of  the  Detailed  Links 
treatment,  even  though  it  was  not  significant  in  that  treatment  In  the  Links  group, 
prior  knowledge  was  not  a  significant  predictor. 

Path  Data 

Coding  Schemes 
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The  computer  recorded  each  screen  viewed,  what  order  they  were  viewed,  and  how 
long  was  spent  on  each  screen.  To  track  student  usage,  a  method  for  classifying  each  move 
from  screen  to  screen  was  developed.  The  method  tracked  the  screen  the  user  was  on 
currently,  the  previous  screen,  and  the  next  screen  chosen  to  determine: 

1)  if  the  user  returned  to  the  previous  screen  or  went  on  to  a  new  one; 

2)  if  the  user  went 

a)  up  the  hierarchy  of  concepts, 
h)  down  the  hierarchy  of  concepts, 

c)  to  a  concept  across  the  hierarchy  at  the  same  level  in  the  same  section, 

d)  to  a  related  concept  in  another  section, 

e)  to  a  concept  before  or  after  the  current  one  in  terms  of  the  flow  of  blood 
through  the  heart, 

f)  back  to  the  objectives  screen; 

3)  what  position  in  the  map  was  clicked  or,  in  treatment  one,  where  in  the  text 
the  clicked  word  was  located; 

4)  how  long  a  user  moved  from  f:«:reen  to  screen  before  returning  to  the  objectives 
screen; 

5)  how  many  screens  following  the  blood  flow  were  linked  together; 

6)  how  many  of  the  sixty  screens  were  viewed; 

7)  how  many  screens  were  seen  more  than  once; 

8)  how  many  screens  were  viewed  in  total; 

9)  how  long  was  spent  on  the.  treatment; 

10)  how  many  of  the  instruction  screens  were  viewed; 

1 1)  how  long  the  user  spent  on  the  instructions;  and 

12)  in  what  order  the  user  chose  to  start  a  new  section  and  how  often  was  each 
started. 

This  information  was  gathered  for  a  random  sampling  of  twelve  subjects  from  each 
of  the  treatment  groups  and  collated  for  each  individual.  The  data  were  examined  in  terms 
of  where  on  the  map  or  in  the  text  the  user  clicked,  what  typical  moves  were  made  by  the 
UHor,  and  how  much  time  was  spent  on  the  program. 

Number  of  Screens  Viewed 

The  program  contained  sixty  screens,  other  than  those  in  the  instructions.  In  the 
entire  sample  of  one  hundred-thirteen  subjects,  the  average  number  of  screens  seen  by  a 
subject  waH  46  (SJ2=12),  and  they  perceived  that  they  saw  about  75%  of  the  screens 
(SJ2=20)  or  42  screens.  They  spent  an  average  of  1257  seconds  or  about  21  minutes  on  the 
treatments  (212=607  seconds).  The  groups  were  largely  the  same  in  the  average  number  of 
screens  seen. 

Moving  Onwards  or  Returning  to  the  Previous  Screen 

Subjects  had  the  choice  of  going  on  to  a  new  screen  from  any  point.,  returning  to  the 
screen  they  had  just  seen,  or  returning  to  the  Objectives  Screen.  The  groups  were  largely 
similar  in  their  movements.  In  general,  the  movement  was  onward  to  a  new  screen  rather 
than  returning  to  the  screen  just  seen.  To  return  to  a  previous  screen,  the  subject  had  to 
remember  which  screen  he/she  just  left  and  then  click  on  the  hotword  or  button  for  that 
screen.  There  was  no  "return"  or  "previous"  button.  Student  ratings  of  feeling  lost  correlated 
with  higher  rates  of  returning  to  the  previous  screen  (r=.46963,  p=.0039)  and  with  a  sense 
of  confusion  instead  of  clarity  (r=.30763,  p=.0682). 
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Uiiaamhkttl  va,  Hierarchical  vs.  Process  Movement 


In  true  hypertext  fashion,  the  user  of  the  program  had  the  ability  to  link  to  a  wide 
variety  of  related  concepts  from  each  screen.  When  the  program  was  constructed,  it  was 
divided  into  four  sections,  Characteristics,  Parts,  Circulation,  and  Blood  Pressure,  that  were 
(ill  linked  to  the  Objectives  Screen.  From  that  screen,  the  user  could  go  down  a  hierarchy  of 
concepts  in  each  section.  The  user  also  had  the  ability  to  move  back  up  the  hierarchy  and 
hack  to  the  Objectives  Screen.  In  addition,  the  user  could  move  to  related  concepts  in  the 
same  section  that  were  across  the  hierarchy  rather  than  up  or  down  it.  They  could  also 
move  to  a  related  concept  in  another  section.  Users  of  the  program  were  free  to  follow  their 
interests,  go  to  familiar  concepts,  or  tackle  new  ideas.  In  the  Hotwords  group,  these  ideas 
were  all  embedded  in  th^  text  and  had  to  merely  be  clicked  to  move  to  a  new  screen.  The 
user  had  to  infer  the  relationships  from  the  other  words  in  the  text.  In  the  Links  group  and 
in  the  Detailed  Links  group  the  users  clicked  buttons  in  the  map  at  the  top  of  the  screen  to 
move  on  to  new  screens.  The  Links  group  merely  had  the  concepts  linked  together  by  lines. 
The  Detailed  Links  group  had  the  nature  of  each  relationship  between  concepts  labeled  on 
the  lines. 

Although  the  Links  group  moved  in  j  slightly  more  hierarchical  fashion  and  chose 
slightly  fewer  heterarchical  links,  the  groups  were  fairly  similar  in  their  movements. 
Overall,  hierarchical  movement  was  the  strategy  of  choice  with  moving  down  the  hierarchy 
the  most  chosen  strategy  followed  by  moving  across  the  hierarchy  within  the  section,  moving 
up  the  hierarchy,  and  moving  across  sections.  The  Hotwords  group  chose  to  move  across  the 
hierarchy  within  sections  more  than  the  other  groups.  This  could  be  a  function  of  a  lesser 
awareness  of  the  existence  of  a  hierarchy  in  the  program.  With  the  high  percental  of 
movement  down  the  hierarchy,  one  might  expact  a  similar  amount  back  up.  but  this  was  not 
the  case.  The  percentage  of  upward  movement  'vas  slightly  more  than  half  of  the 
percentage  of  downward  movement. 

Users  tended  to  start  with  the  fL-st  rone  pt  on  the  Objectives  screen  and  "AOxk  their 
way  through  the  sections  in  order.  They  follw  e  \  stripqa  of  screens  ranging  fron;  c  ^  to  over 
seventy  screens  in  a  row,  but  most  followed  fewer  Ami  thirty  screens  before  returning  to  the 
Objectives  screen. 

Another  possible  screen  selection  strategy  was  to  follow  the  flow  of  th*?  blood  through 
the  heart  as  one  went  from  screen  to  screen.  Both  the  previous  location  and  the  ensuing 
location  were  connected  to  eanh  screen.  Subjects  chose  this  strategy  less  then  12%  of  the 
time  and  most  of  these  choices  were  following  it  only  for  a  screen  ot  two.  A  button  or 
hotword  on  each  screen  led  to  the  part  of  the  heart  immediately  before  and  alter  the  current 
point  in  the  blood  flow.  Only  in  the  Detailed  Links  group  was  thiri  relationship  explicitly 
shown,  although  those  in  the  other  two  groups  could  determine*  this  through  reading  the  text 
itself.  The  patterns  of  screen  selection  following  the  blood  flow  do  not  diffor  greatly  among 
the  groups. 

Click  Locations 

Another  explanation  for  movements  though  the  screens  was  that  students  selected  the 
screen  to  go  to  next,  not  by  the  content,  but  by  the  placement  of  the  hotword  in  the  text  or  the 
bu<ton  on  the  map.  Subjects  in  the  Hotwords  group  did  tend  to  choose  words  near  the 
beginning  of  the  text.  The  low  numbers  at  the  bottom  of  the  Hat  are  partially  a  function  of  the 
few  screens  that  had  that  many  words.  There  is  not  an  indication  that  they  chose  only  the 
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first  word,  so  placement  alone  does  not  appear  to  explain  choice  of  screens. 

Subjects  in  the  Links  and  Detailed  Links  groups  used  the  map  at  the  top  of  the  screen 
to  navigate.  The  word  at  the  top  of  the  map  was  always  the  cui^ept  one  step  up  in  the 
hierarchy.  Only  concepts  directly  connected  to  the  current  screen        shown.  The  other 
terms  around  the  current  concept  button  were  not  necessarily  arrayed  in  a  hierarchical 
fashion,  although  related  terms  were  placed  symmetrically  (e.g.,  Right  Auricle  was  opposite 
Left  Auricle  under  Auricles).  The  two  groups  generally  chose  locations  in  the  same 
proportions.  The  first  counterclockwise  position  (upper  left  corner)  was  chosen  most  often, 
followed  by  th?,  top  position  which  was  one  step  up  in  the  hierarchy.  This  follows  the  research 
that  neople  in  Western  cultures  tend  to  work  from  the/Upper  left  corner  of  a  screen  to  the  lower 
right.  The  first  clockwise  position  (upper  right)  was  chosen  third  most  often  followed  by  the 
second  clockwise  position  and  then  the  second  counterclockwise  position.  Most  screens  had  at 
least  these  positions,  so  these  generalizations  probably  enn  !    .natit1.  Anything  beyond  this  is 
largely  a  function  of  how  often  that  position  appeared  in  the  ,.?up,  It  does  appear  that 
position  was  a  factor  in  selection  of  vhe  ni»xt  concept., 


Attitudes  vnd  Experiences 

Before  logging  off  the  computer  in  session  one,  each  participant  completed  a  survey  on 
the  computer  concerning  his/her  previous  computer  experience,  perceptions  of  moving  through 
the  program,  and  preferred  learning  strategies.  Most  items  were  5-point  Likert-type  scales 
(l=low,  5=high). 

The  majority  of  the  students  did  not  feel  computers  were  very  important  in 
their  daily  lives  and  tew  used  computers  daily.  Students  rated  the  use  of  hypertext  for 
learning  as  moderately  confusing.  Students  indicated  that  they  found  the  program  somewhat 
overwhelming.  This  feeling  appeared  similar  across  all  groups.  Related  to  this  overwhelming 
feeling,  many  students  im-icated  that  it  was  relatively  easy  to  get  lost  (M=3.37).  No  one 
group  appeared  to  have  this  feeling  more  than  any  other.  On  the  other  hand,  the  Links  group 
appeared  to  find  the  program  slightly  more  effective  than  the  other  groups.  Students  felt  the 
program  was  harder  than  text  (M=3.21)  with  the  Hotwords  and  Detailed  Links  groups 
especially  leaning  hi  that  direction. 

Students  were  asked  to  comment  on  the  program  as  an  educational  strategy. 
Of  the  59  who  chose  to  respond,  fc»uceen  commented  favorably,  noting  the  ability  to  go  back 
for  review  and  self  pacing  as  positive  aspect?.  Several  students  commented  that  this  new  form 
of  presenting  information  takes  some  getting  used  to.  Many  described  the  experience  as 
confusing,  frustrating,  overwhelming,  and  long.  Some  felt  it  provided  too  many  options  and  no 
logical  order.  One  student  stated,  "...it  bettered  me  that.it  was  not  easy  to  see  the 
hierarchical  structure  behind  the  materia ..  Osviously  rome  material  would  be  more 
important  than  others  in  any  given  less  hi.  I  found  that  this  program  seemed  to  make  the 
basic  structure  unclear."  A  number  of  others  found  the "  eart  content  boring  or  too  technical. 
Many  felt  they  missed  some  screens  and  easily  got  lost.  In  contrast  to  earlier  statements, 
some  found  it  impossible  to  use  for  review  because  the  navigation  was  very  difficult.  A  few 
expressed  a  general  dislike  of  working  with  computers  and/or  a  preference  for  textbooks. 

Searching  Strategies 

Students  were  asked  to  estimate  how  much  of  the  program  they  had  covered.  A 
counter  indicating  how  many  of  the  60  different  screens  a  student  had  seen  was  located  in  the 
upper  right  corner  of  each  screen.  A  Person's  Product  Moment  Correlation  was  performed  on 
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the  number  of  different  screens  and  the  percentage  estimated  seen.  There  was  only  a 
correlation  of  .35494  (p=. 0337)  between  the6e  two  measures,  indicating  the  students  did  not 
take  full  advantage  of  this  counter  and  had  no  intuitive  idea  of  the  size  of  the  program. 

All  students  indicated  they  sometimes  did  not  go  to  screens  they  already  felt  they 
knew.  Few  indicated  a  willingness  to  skip  such  screens  often.  This  may  indicate  the 
students  either  had  little  knowledge  of  the  topic,  could  not  judge  what  was  on  the  next  screen, 
or  were  unwilling  to  miss  anything.  Students  indicated  they  sometimes  w«nt  to  concepts 
familiar  to  them  which  may  also  indicate  a  hesitancy  to  miss  anything.  Interesting,  there  was 
a  positive  correlation  between  amount  of  prior  knowledge  and  number  of  screen  seen 
(r=  28031,  p=.0977).  Students  with  higher  prior  knowledge  had  a  slight  teudency  to  view 
more  screens.  Students  felt  sometimes  they  viewed  screens  they  had  seen  before,  often 
unwittingly,  but  also  felt  they  sometimes  missed  screens  that  they  skipped  and  then  never 
could  find  again. 

Almost  half  (10  out  of  22)  admitted  to  having  some  problems  with  the  lesson  such  as 
confusion  due  to  flipping  through  so  many  screens,  problems  with  so  many  options,  and 
trouble  finding  missed  screens  to  an  inability  getting  back  to  where  one  wanted  to  go. 
Students  expressed  their  difficulties: 

It  was  kind  of  confusing.  There  were  several  choices  each  time  you  went, 
and  every  time  you  picked  a  choice,  then  it  would  take  you  off  in  a  totally 
different  direction  then  you  were  looking  at  to  begin  with.  (Hotwords  group) 

Sometimes  I  couldn't  get  back  or  had  a  hard  time  getting  back  to  where  I 
wanted  to  get  back  to.  The  checks  were  a  good  idea.  Otherwise  I  would 
have  gone  back  to  things  two  or  three  times  by  mistake.  (Links  group) 

The  navigational  techniques  used  and  the  degree  of  learner  control  were  new  to  most 
students,  and  many  had  difficulty  adapting.  The  majority  of  the  students  (i.e., 
eighteen  out  of  twenty-two)  did  not  find  the  navigational  technique  distracting. 
Some  felt  it  gave  the  main  points  to  be  learned.  As  one  student  put  it: 

It  gave  me  an  idea  of  what  the  main  points  were,  some  of  the  more 
important  information,  so  I'd  look  at  those.  I'd  read  through  the  text  and 
then  go  back  to  the  one  I'd  have  the  most  trouble  with  or  that  would  interest 
me  the  most.  Then  I'd  go  back  and  fill  in  the  gaps  and  read  the  stuff.  I 
wasn't  too  sure  of.  (Hotwords  group) 

A  few  found  it  confusing.  Two  stated  they  read  the  text  and  then  looked  at  the  map, 
but  one  felt  it  made  her  want  to  move  on  without  reading  the  text. 

When  asked  about  the  strategy  they  developed  to  move  from  screen  to 
screen,  the  majority  said  they  had  a  strategy,  although  their  definitions  of  a  strategy 
varied.  Students  based  their  selections  on  prior  knowledge,  on  an  attempt  to  cover 
all  screens,  by  relationship,  or  by  location*]  cues  rather  than  content  oriented  cues. 

When  describing  the  development  of  these  navigational  strategies,  most 
tended  to  start  with  one  technique  and  stick  with  it.  They  tended  to  use  the 
asterisks  or  checks  indicating  screens  already  seen  to  determine  what  they  had 
covered  and  went  back  to  the  Objectives  screen  to  restart  when  they  hit  a  dead  end 
or  got  lost.  Many  never  really  figured  out  how  to  move  back  through  screens. 

When  asked  if  they  skipped  screens  on  concepts  they  already  knew,  nine 
admitted  doing  that  with  three  more  saying  they  skipped  just  a  few.  Ten  never 
skipped  any  screens.  Many  said  they  wanted  to  look  at  all  the  screens.  Some  said 
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they  went  over  those  they  knew  more  quickly,  and  a  few  began  to  skip  or  quickly 
skim  screens  when  they  realized  they  knew  some  of  the  information.  Many  said 
they  went  to  the  familinr  sereeiiH  first  and  then  tried  to  relate  the  new  screens  to  the 
f  amiliar.  Some  of  thm  henitanry  to  Hkip  Hcreens  may  have  been  due  to  the  fact  that 
the  subject  wan  new  to  mowt  subject h. 

Twenty  of  the  HUbjeetH  interviewed  said  they  reread  screens  by  choice, 
especially  if  they  did  not  underntand  the  topic  originally.  Twelve  of  the  subjects  said 
they  reread  screens  by  accident. 

Learner  control  was  a  concept  fairly  new  to  most  of  the  subjects.  They  were 
evenly  divided  on  a  preference  for  learner  control  or  lesson  control.  Those  that  liked 
learner  control  said  they  liked  the  choice  so  they  could  skip  what  was  known,  review 
as  necessary,  and  focus  on  interests.  They  felt  is  involved  them  more  in  learning. 
Those  that  favored  lesson  control  said  it  provides  necessary  organization  so  the 
learner  knows  what  is  important,  doesn't  waste  energy  in  organizing  the  content, 
and  provides  a  logical  order  for  the  information.  Several  felt  lost  navigating  oh  their 
own.  Others  felt  they  might  miss  important  information,  especially  on  a  subject  new 
to  them. 

Those  in  the  Detailed  Links  group  were  asked  about  their  use  of  the  labels 
in  the  graphical  browser.  Five  of  the  seven  said  they  read  the  labels,  although  most 
said  they  did  not  do  it  all  of  the  time.  If  this  is  the  case,  subjects  in  this  group  and 
subjects  in  the  Links  group  were  often  performing  in  the  same  fashion,  thereby 
minimizing  the  effects  of  the  two  treatments. 

At  the  beginning  subjects  had  been  asked  to  think  of  the  lesson  as  a  new 
way  of  learning.  They  were  asked  to  list  the  good  and  bad  points  of  this  way  of 
learning.  Good  points  included  1)  it  was  different  and  interesting;  2)  it  provided 
many  choices  and  the  chance  to  skip  or  review  as  desired;  3)  it  provided  learner 
control  to  choose  when  and  in  what  order  screens  were  read;  4)  it  interactively 
involved  the  learner  and  was  less  boring;  5)  it  gave  the  structure  of  the  information 
in  the  map;  6)  it  included  useful  diagrams;  7)  there  was  a  small  amount  of 
information  on  each  screen;  and  8)  it  was  less  time  consuming  than  reading.  Some 
of  these  strengths  were  weaknesses  to  other  subjects.  Negative  aspects  of  this  way 
of  learning  mentioned  included:  1)  it  allowed  a  lot  of  skipping  around;  2)  navigation 
was  hard,  often  confusing,  and  difficult  to  move  back  and  forth;  3)  it  did  not  provide 
a  concrete  order  for  the  information  or  emphasize  the  most  important  information;  4) 
it  is  difficult  for  those  with  little  background  knowledge;  5)  use  of  the  computer  itself 
was  distracting;  6)  it  was  possible  to  go  through  it  quickly  and  not  learn  the 
information;  7)  it  was  hard  to  keep  track  of  what  was  covered;  8)  it  was  not  an 
interesting  topic;  9)  the  amount  of  information  was  overwhelming;  and  10)  not 
everyone  has  access  to  a  computer.  Overall,  students  felt  there  were  possibilities  for 
this  type  of  learning  but  it  was  confusing  and  did  not  provide  enough  structure  for 
the  information. 

Limitations  of  the  Study 

Testing  was  a  threat  in  this  study.  The  immediate  and  delayed  posttests 
were  alternate  versions  of  the  same  test  possibly  causing  some  improvement  in  the 
delayed  test  due  to  the  subjects1  experience  with  the  immediate  posttest.  Another 
threat  to  internal  validity  was  the  method  for  assessing  structural  knowledge.  Most 
students  are  not  familiar  with  such  assessment  methods  and  have  trouble  carrying 
them  out.  Mean  scores  for  the  task  were  only  17  out  of  44  (SH=13)  before  the 
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treatment  and  19  (SQ=13)  two  weeks  after  the  treatment.  Some  students  produced 
trees  with  no  or  very  low  structure  either  before  or  after  the  treatment. 

A  low  correlation  was  found  between  the  posttreatment  ordered-tree 
measuring  the  structure  of  a  learner's  content  information  and  the  immediate  and 
delayed  criterion-referenced  tests  (r=.33156  for  the  immediate  posttest,  r=.36401  for 
the  delayed  posttest).  This  is  another  possible  indication  that  the  orderedtree  task 
did  not  differentiate  well  between  the  subjects.    The  question  of  whether  focusing 
on  the  concepts  and  their  relationships  helps  or  hinders  recall  of  facts  has  emerged 
in  the  research.  Jonassen  and  Wang  (1990)  showed  that  stressing  this  type  of 
learning  may  actually  hinder  factual  recall.  In  the  present  study,  the  students  did 
not  do  worse  on  the  posttests  than  in  similar  studies  with  print  materials,  but  the 
structural  knowledge  tasks  did  not  show  a  large  acquisition  of  this  type  of 
knowledge. 

The  comparisons  to  an  "expert"  structure  were  conducted  using  the  structure 
of  the  content  in  the  lesson.  This  "expert"  tree  was  constructed  by  the  instructional 
developer  and  should  not  be  considered  that  of  an  expert  in  the  field,  but  rather  that 
represented  in  the  lesson.  It  was  verified  by  a  high  school  biology  teacher  and  a 
nurse  for  accuracy.  The  similarity  ratings  must  be  interpreted  with  caution  as  there 
arfe  numerous  valid  ways  to  structure  this  information.  Each  individual  may  go 
through  the  lesson  and  construct  different  personal  structures. 

The  ordered-tree  technique  also  may  have  influenced  the  study  by  alerting 
subjects  to  important  concepts.  As  these  terms  were  not  all  that  was  evaluated  on 
the  posttest,  they  may  have  provided  a  false  sense  of  security  in  selecting  what  to 
study.  Many  seemed  to  put  little  effort  into  it  at  the  time  of  the  delayed  posttest. 

Experimental  mortality  was  a  problem,  with  twelve  students  not  completing 
the  first  session,  ten  not  returning  to  the  second,  and  five  dropped  due  to  lack  of  SAT 
scores.   Even  with  this  problem,  the  groups  ended  up  with  almost  equal  numbers, 
although  it  is  not  known  if  the  students  lost  were  randomly  distributed  across  all 
independent  variables. 

Generalizabiiity  or  external  validity  threats  were  also  present  in  this  study, 
most  notably  the  fact  that  volunteers  who  did  this  for  extra  credit,  not  as  part  of 
class  content,  were  used. 

Another  problem  was  the  length  of  treatment.  The  idea  was  to  work  within 
a  constrained  subject  domain  so  that  all  the  subjects  had  the  possibility  of  going 
through  all  the  material  Students  not  interested  in  a  scientific  topic  may  have 
found  even  the  time  spent  boring.  Twelve  subjects  chose  to  view  only  one  or  two 
screens  before  exiting.  At  the  same  time,  the  experience  with  the  different  types  of 
structural  knowledge  roprcHentations  may  not  have  been  enough  to  see  an  effect,  nor 
were  the  students  actively  involved  in  generating  the  structural  frameworks. 

Multiple  regression  was  UHed  for  exploring  the  data  in  this  study.  For  the 
equation  estimated  for  the  immediate  postteit  the  multiple  correlation  coefficient 
was  fairly  high,  accounting  for  sixty  percent  of  the  variance  in  the  dependent 
variable,  but  for  the  other  equations  the  multiple  correlation  coefficient  was 
between  thirty  and  thirty-five,  accounting  for  only  that  percentage  of  the  variance  in 
the  dependent  variable.  The  low  multiple  correlation  coefficients  could  be  due  to 
omitting  relevant  predictor  variables,  including  irrelevant  variables,  and/or  using 
variables  in  an  incorrect  form  when  the  relationships  are  p.fci  -linear.  All  are  possible 
in  this  study  with  the  most  likely  being  omission  of  relevant  variables.  Other 
individual  cognitive  differences,  experience  with  hypertext,  risk-taking  behavior,  and 
motivational  factors  might  be  likely  predictors  of  performance. 
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Conclusions  and  Recommendations 

This  study  suggests  that  users  of  hypertext  require  extended  experience  with 
such  a  system  to  become  comfortable  and  proficient  using  it.  Graphical  browsers 
may  provide  a  degree  of  structure  for  the  user,  but  it  was  not  evident  from  the 
results  that  all  learners  internalized  this  structural  knowledge.  Those  with  high 
prior  knowledge  did  better  on  most  variables  and  showed  a  greater  increase  in 
structural  knowledge.  The  use  of  hotwords  rather  than  a  graphical  browser  resulted 
in  lower  achievement  on  the  criterion  test  dependent  measures  and  was  a  special 
problem  for  those  with  lower  prior  knowledge.  Verbal  ability  was  not  a  factor  on 
most  of  the  variables.  Further  research  incorporating  extended  hypertext  use  may 
help  to  determine  the  best  methods  for  aiding  learners  to  acquire  knowledge  in  a 
hypertext  system. 

Effect  of  Using  Different  Navigational  Techniques 

On  both  the  immediate  posttest  and  the  change  between  immediate  and  delayed 
posttests  treatment  was  a  significant  predictor.  The  Hotwords  group  was  forced  to  generate 
their  own  framework  for  the  knowledge  because  the  words  were  embedded  in  the  text,  not 
arranged  relationally.  It  was  this  group  that  consistently  performed  the  lowest  on  all 
dependent  measures.  This  group's  performance  was  also  the  most  influenced  by  degree  of 
prior  knowledge,  most  notably  on  the  structural  knowledge  dependent  variables.  The  Links 
and  Detailed  Links  groups  who  used  the  two  different  forms  of  the  graphical  browser 
performed  similarly  on  several  of  the  dependent  measures,  perhaps  due  to  the  members  of 
the  Detailed  Links  group  who  did  not  consistently  read  the  labels  in  the  graphical  browser. 

Additional  individual  subject  variables  interacted  with  the  treatments,  indicating 
that  no  one  treatment  was  best  for  all  subjects.  The  contribution  of  these  variables  differed 
on  the  various  dependent  measures. 

Influence  of  Prior  Knowledge 

Prior  knowledge  was  a  significant  predictor  of  immediate  posttest  score,  the 
difference  between  delayed  and  immediate  posttest  scores,  and  the  difference  between 
pretreatment  and  posttreatment  ordered-tree  task  scores.  It  was  not  a  significant  predictor 
of  the  posttreatment  ordered-tree  task  score  itself. 

Prior  knowledge  did  not  interact  significantly  with  the  treatment  on  any  of  the 
dependent  measures  except  the  posttreatment  ordered-tree  task.  On  this  variable  it  was 
significant  only  in  the  Hotwords  group.  It  was  significant  only  in  the  Hotwords  group  for 
dependent  variables  representing  the  difference  between  posttest  scores.  This  may  indicate 
that  learning  using  the  Hotwords  treatment  which  was  less  obvious  in  its  portrayal  of  the 
content  structure  was  more  diflicul'  "or  students  with  lower  prior  knowledge  as  evidenced  by 
several  of  the  dependent  measures.  The  treatments  that  provided  more  structural, 
knowledge  support  appeared  to  be  less  influenced  by  degree  of  prior  knowledge.  This  study 
suggests  further  research  on  methods  for  compensating  for  low  prior  knowledge  is 
warranted. 

In  an  examination  of  the  path  information,  students  with  higher  prior  knowledge 
tended  to  view  more  of  the  total  number  of  screens.  This  may  reflect  a  greater  interest  in 
the  topic  or  the  general  unwillingness  to  skip  screens,  even  if  the  information  was  known. 
Those  with  lower  prior  knowledge  may  have  experienced  more  frustration  and  exited  the 
lesson  more  quickly.  Research  on  learner  control  has  shown  that  learners  often  do  not  know 
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what  is  best  for  their  own  learning,  and,  even  if  they  do,  may  not  act  on  this  knowledge 
(Milheim  &  Martin,  1991;  Jonassen,  1986).  This  is  a  special  problem  for  those  with  low 
prior  knowledge  (Steinberg,  1977;  Milheim  &  Martin,  1991).  Even  college  students  have 
difficulty  with  this  self-assessment  (Garhart  &  Hannafin,  1986). 

In  the  interviews,  less  than  half  said  they  skipped  any  of  the  screens  they  already 
knew  and  of  those,  most  skipped  only  a  few.  More  common  was  the  tendency  to  look  at  a 
screen  and  go  over  it  quickly  if  the  content  was  already  known.  A  number  of  students 
indicated  they  used  their  prior  knowledge  to  begin  with  familiar  screens  and  then  tie  new 
screens  to  this  known  information. 

Comparisons  of  subjects'  and  the  "expert"  ordered-trees  showed  no  significant 
influence  of  prior  knowledge,  although  it  approached  significance  in  the  Hotwords  group  and 
appeared  in  a  general  positive  direction  in  the  Detailed  Links  group. 

Influence  of  Verbal  Ability 

Verbal  ability  was  not  a  major  influence  on  most  of  the  dependent  measures, 
possibly  due  to  its  correlation  with  the  independent  variable  Prior  Knowledge.  Only  in  the 
difference  between  the  immediate  and  delayed  posttests  did  it  prove  significant,  and  then 
only  in  interaction  with  the  treatment. 

Influence  of  Time  on  Treatment  and  Number  of  Screens  Seen 

Time  spent  on  treatment  and  number  of  screens  seen  contributed  to  the  variance  in 
scores  on  the  immediate  posttest  only.  Treatment  group  interacted  with  both  variables  no 
that  subjects  in  the  three  groups  performed  differently  on  the  immediate  posttest  as 
influenced  by  these  two  variables.  Number  of  screens  seen  contributed  significantly  only  in 
the  Links  and  Detailed  Links  groups,  approaching  significance  in  the  Hotwords  group,  while 
time  spent  on  treatment  contributed  significantly  only  in  the  Hotwords  group. 

Many  subjects  did  not  view  all  the  screens  due  to  fatigue  or  boredom,  a  choice  of 
degree  of  effort  to  invest,  or  accidentally  missed  screens  and  an  inability  to  find  them  again. 
Subjects  viewing  significantly  under  the  sixtyscreens  may  not  have  covered  all  the 
information  evaluated  in  the  posttest.  The  number  of  screens  seen  made  a  difference  on 
posttest  scores  for  the  Links  and  Detailed  Links  group,  but  not  for  the  Hotwords  group 
where  there  was  a  very  slight  negative  influence  from  the  number  of  screens  seen. 

Time  spent  could  represent  either  persistence  in  finding  all  the  Hcrcun*  and/or 
amount  of  time  spent  reading  the  screens.  The  latter  may  have  been  important  only  in  the 
Hotwords  treatment  because  more  persistence  may  have  been  required  to  actually  sue  all 
the  screens.  The  screens  covered  were  marked  with  asterisks  within  the  text  itHulf,  but  this 
indication  may  not  have  been  as  obvious  to  the  subjects  as  the  checkmarks  used  in  the 
graphical  browsers.  It  was  surprising  that  those  at  the  low  end  of  the  time  viiriiible  did  an 
well  as  they  did  on  the  test,  considering  they  spent  such  a  small  amount  of  time  viewing  the 
screens.  Some  of  the  elapsed  time  was  spent  on  navigation,  as  a  number  of  Htudents  spent 
the  last  few  minutes  looking  for  screens  they  had  missed  rather  than  reading  now  material. 

The  effect  of  these  two  variables  was  not  evident  in  the  change  from  immediate  to 
delayed  posttest.  Simply  viewing  the  more  screens  quickly  might  influence  immediate  recall 
of  the  information,  but  not  allow  for  integration  of  this  information  into  long  term  memory. 
Viewing  more  screens  did  not  translate  into  a  proportionally  long  time  spent  on  the 
treatment. 

Changes  in  Structural  Knowledge 
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The  posttreatment  structural  knowledge  scores  in  this  study  did  not  rise  from  the 
pretreatment  scores  for  all  subjects  nor  did  they  necessarily  grow  closer  to  an  "expert" 
structure  reflected  in  the  lesson.  Those  with  high  initial  structural  knowledge  showed  the 
highest  posttreatment  scores,  especially  in  the  Detailed  Links  group.  It  is  conceivable  that 
some  of  these  students  may  have  used  the  pretreatment  task  as  an  organizer  for  the  new 
information,  particularly  because  the  graphical  browsers  in  this  group  spatially  portrayed 
some  of  the  same  terms  used  in  the  task  and  indicated  their  relationships. 

Those  who  showed  greater  pretreatment  structural  knowledge  actually  did  not  gain, 
but  lost  structural  knowledge  from  the  pre  to  posttreatment  task.  Some  of  this  may  be 
attributed  to  the  statistical  phenomenon  of  regression  to  the  mean.  There  may  also  have 
been  some  problem  with  degree  of  effort  expended  on  the  posttreatment  task.  Another 
posHible  explanation  could  be  that  some  students  who  had  a  structure  before  the  task  were 
caused  to  question  that  structure  after  exposure  to  the  structure  in  the  lesson  but  never 
integrated  thin  new  structure  into  their  schema.  On  the  other  hand,  those  who  did  better  on 
the  immediate  posttcst  tended  to  demonstrate  more  similarity  to  the  "expert"  ordered-tree, 
imperially  in  the  Links  and  Detailed  Links  groups.  This  reaffirms  the  idea  that  degree  of 
structure  is  nut  synonymous  with  similarity  to  the  "expert"  structure.  Students  may  have 
lost  some  of  their  original  structure  which  was  unlike  the  one  in  the  lesson  and  moved  closer 
to  that  portrayed  in  the  lesson,  but  never  reaching  the  same  degree  of  structure  shown 
initially, 

Further  research  is  needed  on  the  relationship  between  declarative  knowledge  as 
measured  in  the  immediate  and  delayed  posttests  and  structural  knowledge  measured  by 
the  ordered-tree  task.  If  declarative  knowledge  is  embedded  in  structural  knowledge,  one 
would  expect  n  closer  correlation  in  all  groups  between  the  immediate  posttcst  and  the 
posttreatment  ordered-tree  and  between  the  delayed  posttest  and  the  posttreatment 
ordered-tree.  Students  who  learned  the  relationships  between  the  concepts  in  the  lesHon 
would  have  also  learned  facts  about  those  concepts.  If,  as  Jonassen  and  Wang  (1992) 
stated,  one  interferes  with  the  other,  e.g.,  the  acquisition  of  the  relationships  between 
concepts  interferes  with  learning  details  about  the  concepts,  a  negative  relationship  would 
be  predicted  between  scores.  In  this  study,  the  correlation  between  the  immediate  posttest 
and  the  posttreatment  structural  knowledge  task  was  only  .33  and  between  the  delayed 
posttest  and  the  posttreatment  structural  knowledge  task  only  .36. 


Problems  in  Learning  from  Hypertext 

In  general,  students  in  all  treatments  expressed  some  confusion  in  going  through  the 
treatments.  Complete  learner  control  over  the  pace  and  path  of  information  seen  was  new 
to  most  of  the  students.  Many  felt  lost  and  frustrated  as  they  tried  to  work  their  way 
through  the  many  options  on  each  screen  and  formulate  their  own  structure.  Only  after 
sustained  use  of  such  a  system  for  a  period  of  time  do  users  become  comfortable  with  this 
control  and  develop  the  strategies  needed  to  make  selections  and  follow  subject  paths 
(Landow,  1990;  Oren,  1990;  Remde,  Gomez  and  Landauer,  1987;  Marchionini,  1988). 

This  stud>' s  subjects  spent  an  average  of  only  twenty-one  minutes  on  the  lesson. 
Many,  in  fact,  spent  as  much  if  not  more  time  on  the  testing.  They  viewed  an  average  of 
forty-four  of  the  sixty  possible  screens.  This  was  not  a  long-term  involvement  with  hypertext 
that  would  be  needed  for  familiarization  with  the  technique.  Multiple  trials  with  navigation 
through  a  hypertext  system  should  better  prepare  the  subjects  to  direct  their  own  learning 
through  such  a  system. 

Hypertext  use  also  can  cause  an  additional  cognitive  burden  by  requiring  users 
remember  the  links  just  made  and  connect  all  this  information  into  a  coherent  whole.  Users 
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also  had  to  discover  and  remember  the  consequence  of  following  a  certain  path,  where  it  led, 
and  types  of  information  it  provided  (Heller,  1990).  Examination  of  path  data  showed  a  few 
students  restarting  the  same  section  a  number  of  times,  only  to  terminate  this  path  quickly, 
possibly  because  he  or  she  decided  it  was  something  previously  covered.  Perhaps  some  way 
of  indicating  what  percentage  of  the  screens  seen  under  any  of  the  four  main  section 
headings  would  help  in  this  decision  making. 

Although  the  counter  in  the  upper  right  corner  indicated  how  many  of  the  total 
number  of  screens  had  been  seen,  there  was  no  indication  where  the  remaining  screens  wore 
located.  This  may  not  be  a  problem  if  the  system  is  used  for  locating  information  for  a 
specific  information  need,  but  students  intent  on  seeing  all  the  screens  in  an  instructional 
situation  became  frustrated  and  jumped  around  searching  for  a  few  missing  screens. 

This  additional  cognitive  load  can  distract  from  the  content  of  the  lesson  (Mayes, 
Kibby,  and  Anderson,  1990).  Only  one  student  interviewed  specifically  mentioned 
distraction  from  the  use  of  the  graphical  browser,  although  several  mentioned  they  found  the 
browser  difficult  to  use  and  found  keeping  track  of  where  they  had  been  a  burden.  The  hick 
of  normal  reading  cues  added  to  the  difficulty  of  using  such  a  system,  especially  for  less 
skilled  readers  (Charney,  1987,  Horn,  1990).  Only  two  of  the  students  interviewed  stated 
that  the  graphical  browsers  consciously  indicated  the  structure  of  the  content  to  them,  and 
just  a  few  said  they  used  that  structure  to  help  them  decide  where  to  go  next.  This  could 
indicate  a  lack  of  understanding  of  the  maps  themselves  or  a  lack  of  attention  to  these  eues 
presented  in  a  fashion  unfamiliar  to  the  students. 

The  sheer  amount  of  information  presented  and  the  number  of  decisions  that  needed 
to  be  made  could  also  have  been  distracting  to  the  user.  A  number  of  those  interviewed 
mentioned  the  difficulty  of  dealing  with  so  many  options  and  this  was  listed  as  a  reason  for 
preferring  lesson  control  by  several.  The  students  were  given  four  major  objectives  on  the 
first  screen  that  formed  the  basis  for  the  organization  of  the  hypertext  screens,  hut  many  did 
not  appear  to  use  these  in  their  searching.  The  lack  of  a  clearly  evident  structured  path 
through  the  screens  may  have  caused  students  who  did  not  develop  a  strategy  to  wander 
aimlessly,  miss  relevant  material,  or  form  a  wrong  interpretation  (Marchionini,  1988). 
Perhaps  with  such  a  short  exposure,  the  subjects  did  not  have  the  chance  to  develop  the 
needed  skills  in  navigating,  absorbing  layers  of  information,  integrating  new  information 
with  existing  knowledge,  gaining  a  comprehensive  picture  of  the  subject,  self-management, 
searching,  and  manipulation  of  the  interface  needed  for  hypertext  systems  (Bourne,  1990; 
Jonassen,  1989b;  Kinzie  &  Berdel,  1990;  Wright,  1990;  Marchionini,  1988). 

Motivation  could  also  have  been  a  problem  in  this  study,  carried  out  with  volunteer 
subjects  who  had  no  stake  in  learning  the  content.  Their  only  extrinsic  motivation  was  to 
show  up  for  two  sessions  to  get  extra  credit  points  as  the  content  was  not  related  to  their 
elasswork.  The  content  of  the  heart  was  not  of  great  interest  to  a  number  of  those 
interviewed.  With  motivation  an  issue  and  the  lack  of  focus  on  the  objectives  given,  some 
students  may  have  merely  rambled  through  the  system  when  given  complete  learner  control 
(Hammond,  1989)  and  may  have  exited  the  program  before  covering  all  the  material. 

Milheim  and  Martin  (1991)  suggested  that  allowing  the  learner  to  control  the 
sequence  of  the  content  may  not  be  effective  when  the  content  has  a  specific  prerequisite 
order.  They  also  suggested  if  all  topics  must  be  covered  to  successfully  complete  the  program 
or  when  topics  have  a  hierarchical  order,  learner  selection  of  content  to  be  covered  is  not 
effective.  The  results  of  this  study  cannot  strongly  support  or  repudiate  these  claims,  but 
the  mean  scores  on  the  criterion-referenced  posttest  were  equal  to  or  slightly  higher  than 
those  in  studies  using  the  linear,  print  form  of  the  materials  when  adjusted  for  test  length 
(e.g.,  Hodes,  1990;  Cardinale,  1990).  No  comparable  measures  of  structural  knowledge  are 
available. 

This  hierarchically  structured  content  may  not  have  been  the  most  appropriate  for  a 
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hypertext  system.  Duchastel  (llltMl)  and  Spiro  and  Jehng  (1990)  have  suggested  that 
hypertext  is  not  appropriate  lor  highly  structured  learning  tasks  where  students  may  benefit 
from  having  that  structure  explicitly  provided  to  them.  Students  are  not  presented  the 
information  in  the  most  logical  sequence,  nor  ure  facts  and  lower  level  concepts  mastered 
before  higher  level  ones  ((itigne,  Hriggs,  &  Wager,  1988),  They  are  left  to  discover  this 
structure  on  their  own,  a  task  especially  difficult  for  those  with  no  prior  knowledge  on  the 
topic.  The  whole  question  of  whether  or  not  the  structure  of  the  system  was  ever  conveyed  to 
the  subjects  must  be  raised.  The  low  mean  scores  on  the  posttreatment  ordered  tree  task 
(M=17)  indicated  either  a  lack  of  understanding  of  the  content  structure,  a  lack  of 
understanding  of  the  task  or  frustration  with  the  task, 

Students  interviewed  expressed  some  difficulty  in  discovering  the  major  concepts 
first.  The  system  was  structured  so  that  the  user  could  move  down  the  hierarchy  of 
concepts,  but  this  may  not  have  been  obvious  to  all.  Users  could  cover  all  the  main  concepts 
first  and  then  delve  into  each  in  more  depth  as  desired.  This  may  be  one  type  of 
recommended  hypertext  use  strategy  that  requires  more  explanation  or  instruction  for  the 
user. 

Learning  the  process  of  the  blood  flow  through  the  heart  was  also  an  aspect  of  this 
lesson.  Such  a  linear  process  can  be  less  obvious  in  a  non-linear  system  unless  it  is  clearly 
delineated.  If  hypertext  is  to  be  used  to  learn  the  steps  in  a  process,  the  steps  need  to  he 
more  clearly  linked  in  the  proper  order. 


Search  Strategies 


Students  were  asked  about  their  search  strategies,  both  in  the  survey  and  in  the 
interviews  and  their  path  data  was  analyzed.  In  the  survey,  they  miid  they  returned  to  (he 
previous  screen,  saw  screens  more  than  once,  and  missed  seeing  some  screens  they  wanted 
to  get  back  to  "sometimes"  to  "often"  on  a  five  point  scale  (i.e.,  never,  sometimes,  oil  en,  most 
of  the  time,  always).  Analysis  of  the  path  data  of  a  sampling  of  thirty-six  subjects  shewed 
that  they  returned  to  the  previous  screen  about  ten  percent  of  the  time,  going  an  to  a  new 
screen  eighty-four  percent  of  the  time  and  returning  to  the  main  Objectives  screen  six  percent 
of  the  time.  A  number  of  students  never  appeared  to  grnsp  the  concept  of  moving  hack 
through  a  path. 

The  largest  percentage  of  movement  in  all  groups  was  down  the  hierarchy  with 
hierarchical  movement  the  most  common  overall.  The  highest  percentage  of  options  in  the 
graphical  browsers  was  down  the  hierarchy,  so  this  was  expected.  The  Links  group 
performed  slightly  more  hierarchical  movements  and  fewer  heterarchical  movements  when 
compared  to  the  other  two  groups.  The  Hotwords  group  tended  to  move  across  sections 
more,  possibly  because  the  structure  was  less  visible,  Only  twelve  percent  of  the  movements 
across  all  groups  followed  the  flow  of  blood  and  much  of  this  was  just  one  screen  to  the  next, 
not  a  long  string  of  screens  following  the  blood  flow.  Students  either  did  not  see  this  as  an 
option  or  chose  to  follow  the  more  obvious  hierarchical  framework  of  the  lesson. 

Movement  across  the  hierarchy  was  largely  confined  to  within  a  section  rather  than 
jumping  across  sections  to  related  concepts,  Even  without  a  strict  hierarchy  indicated  in  the 
navigational  technique,  students  tended  to  follow  a  natural  hierarchy,  although  some 
students  indicated  in  the  interviews  that  they  had  a  hard  time  perceiving  the  structure  of 
the  content.  In  learning  a  process  such  as  the  flow  of  blood  through  the  heart,  this 
hierarchical  movement  may  be  most  useful  for  learning  the  parts  of  the  heart,  but  not  for 
learning  the  actual  process.  If  students  are  to  follow  this  blood  flow,  it  must  be  more  clenrly 
delineated  in  the  choices  for  navigation. 

Where  the  students  clicked  on  the  screen  was  also  recorded  to  determine  if  actual 
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placement  on  the  screen  was  a  factor  in  choosing  where  to  go  next  rather  than  the  actual 
content  itself.  Those  in  the  Hotwords  treatment  tended  to  choose  the  top  three  words  in  the 
text,  although  not  selecting  the  top  term  any  more  often  than  the  others.  The  groups  with 
the  graphical  browser  tended  to  select  the  term  in  the  upper  left  corner  most  often,  followed 
by  the  term  at  the  top  then  the  term  in  the  upper  right  corner.  This  may  indicate  the 
tendency  to  start  reading  in  the  upper  left  corner  and  proceed  to  the  lower  right.  In  general, 
the  terms  on  the  left  side  of  the  graphical  browser  were  selected  more  often  that  parallel 
terms  on  the  right  side.  Placement  on  the  map  may  be  just  as  important  in  selection  of  the 
next  screen  as  content.  If  the  hierarchy  were  more  evident  in  the  browser,  perhaps  this 
would  be  less  the  case. 


Implications  for  Instructional  Design 

No  one  technique  proved  best  for  all  students.  When  compared  to  other  studies  by 
Dwycr  using  the  same  materials  in  a  written  form,  the  scores  on  the  criterion-referenced 
posttostH  were  not  greatly  improved,  although  results  were  better  than  some  dissertation 
studies  using  print  versions  of  the  heart  materials  (e.g.,  Hodes,  1990;  Cardinale,  1990),  nor 
time  saved  by  this  instructional  method  (Dwyer,  personal  correspondence).  No  comparisons 
on  the  structural  knowledge  task  were  available.  Individual  differences,  such  as  prior 
knowledge,  accounted  for  much  of  the  variance  in  learner  achievement. 

In  general,  the  use  of  this  hypertext  system  was  not  intuitive  for  first-time  users. 
Students  not  used  to  this  degree  of  learner  control  often  felt  lost  and  confused.  Many  had 
trouble  developing  a  viable  strategy  for  moving  through  and  organizing  the  information. 
Many  used  the  indicators  of  screens  already  seen  (i.e.,  asterisks  or  checkmarks)  and  number 
nf  different  screens  shown  in  the  upper  right  corner,  but  still  had  trouble  moving  back 
through  a  path  and  selecting  from  the  many  options  on  a  screen. 

Several  suggestions  are  made  for  structuring  a  hypertext  system  for  learning. 

1 .  Results  suggest  that  the  graphical  browser  would  be  a  better 
navigational  tool  than  the  use  of  the  Hot  words  alone,  especially  for 
those  with  lower  prior  knowledge. 

2.  Students  had  difficulty  getting  back  to  where  they  came  from. 
Techniques  such  as  path  histories  of  recent  screens,  an  indication  of 
the  immediate  previous  screen,  a  thumbnail  of  the  previous  screen,  or 
a  "return"  arrow  to  go  to  the  previous  sereenrnight  be  useful. 

ii.       The  graphical  browser  could  he  structured  to  better  represent  the 
types  of  relationships.  Terms  up  and  down  the  hierarchy  might  be 
differentiated  from  those  that  jump  to  other  sections  or  those  that 
follow  the  flow  of  blood. 

4.  A  global  map  or  web  view  showing  areas  covered  and  relationships 
beyond  the  immediate  screen  and  its  direct  links  could  be  made 
available  with  the  ability  to  zoom  in  or  out  to  a  global  map. 

5.  Some  way  of  indicating  what  percentage  of  the  screens  seen  under  any 
of  the  four  main  section  headings  would  help  in  deciding  how  much  of 
each  section  has  been  covered. 

6.  Not  all  students  work  well  with  a  graphical  representation  of  the 
content  structure  or  prefer  this  method  of  navigating,  so  providing 
some  type  of  menu  listing  might  be  more  appropriate  for  these 
users, It  might  be  possible  to  give  the  user  a  choice,  hut  this  decision 
may  be  difficult  to*  the  naive  user. 
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7.       Users  with  low  prior  knowledge  or  those  with  little  experience  with 
this  type  of  system  may  need  additional  guidance  in  their  initial  use 
of  the  system.  Possibly  a  type  of  guided  tour  which  would  highlight 
the  major  concepts  to  be  covered  first  would  be  useful  for  an  overview 
(Oren,  1990).  Such  guidance  could  be  available  on  demand  in  a  menu 
or  pop-up  window  to  recommend  a  path,  next  step(s),  and/or  strategy. 
Novice  and  expert  users  of  the  system  may  prefer  different  degrees  of 
guidance. 

H.  Some  additional  method  for  getting  to  missed  screens  is  needed, 
perhaps  integration  of  information  on  missed  sections  or  screens  into 
an  overview  map  or  diagram. 

9.  A  keyword  search  capability  would  be  useful  for  those  looking  for  a 
specific  piece  of  information. 

10.  Hypertext  on  its  own  is  not  necessarily  instructional.  Such  a  program 
might  be  best  used  as  part  of  a  large  instructional  system  where 
learners  are  either  given  or  develop  specific  objectives  for  using  the 
system. 

1 1.  Highly  structured  information  which  might  be  easiest  to  learn  in  a 
specified  order  may  best  be  taught  in  another  fashion.  Processes  may 
also  benefit  from  a  more  structured  system  with  more  limited 
hypertext  features. 

12.  An  online  notebook  could  be  used  for  taking  notes,  copying  information 
from  the  screen,  incorporating  prior  knowledge  and  information  from 
other  sources,  and/or  keeping  track  of  the  knowledge  structure  or  path 
covered. 

13.  Students  interviewed  did  not  show  a  desire  to  provide  their  own  labels 
on  the  graphical  browsers,  but  this  generative  activity  might  be 
included  as  part  of  the  instructional  system.  It  would  also  draw 
students'  attention  to  the  graphical  browser  and  its  structural 
knowledge  content. 

Suggestions  for  Future  Research 

Thin  was  an  exploratory  analysis  designed  to  determine  some  of  the  important 
variablea  in  the  unn  of  hypertext  systems  for  learning.  Variables  such  as  prior  knowledge 
and  number  of  screens  seen  warrant  inclusion  in  future  studies.  Several  suggestions  for 
future  research  fellow. 

I .  Improve  the  use  of  the  ordered-tree  technique  for  assessing  structural 
knowledge  or  explore  other  possibilities  for  this  assessment,  Students 
need  greater  familiarity  with  the  technique  and  practice  in  its  use 
beforehand. 

2.  Compare  the  acquisition  of  structural  knowledge  in  this  technique  to 
using  the  print  form  of  the  lesson.  If  there  is  no  gain  on  any  of  the 
dependent  measures  using  the  hypertext  version,  there  may  he  little 
reason  to  invest  the  effort  in  its  development  unless  skills  such  as 
information-seeking  and  comprehension  monitoring  in  a  non-linear 
environment  are  to  be  developed. 

3.  Using  the  same  basic  research  design,  expand  the  study  for  use  in  a 
real  life  setting  with  content  appropriate  for  class  use. 

4 .  Further  explore  the  use  of  different  types  of  graphical  browsers. 

5.  Incorporate  further  investigation  of  search  strategies  both  through 
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qualitative  data  and  analysis  of  path  data  in  long-term  studies. 
Comparisons  of  novice  and  export  users  could  help  describe  the 
development  of  usahle  strategies. 

6.  Assess  motivational  and  attitudinal  issues  in  the  UHe  of  these  systems 
as  well  as  other  individual  differences  such  as  risk-taking  behavior, 
field  articulation,  and  metacognitivo  ability.  Ixiok  at  specific  aspects 
of  reading  ability  that  may  affect  hypertext  use. 

7.  Look  at  the  relationship  between  structural  knowledge,  factual 
knowledge,  and  problem  solving. 

8.  Study  the  use  of  hypertext  systems  for  learning  in  younger  students 
and  the  cognitive  demands  of  such  a  systems. 

9.  Investigate  other  navigational  methods  for  using  hypertext  systems 
and  their  relationship  to  structural  knowledge.  Study  how  these 
systems  can  replicate  or  replace  traditional  reading  cues. 
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Two  significant  events  are  underway  in  Iowa.  First,  the  Iowa  Communications  Network 
(ICN),  a  statewide,  full-motion  interactive  fiber  optic  network,  is  uenring  completion.  The 
ICN  is  a  2800  mile  system  that  connects  sites  in  each  of  Iowa's  99  counties  with  a  fiber 
optics  network  for  distance  education.  The  ICN  is  being  funded  entirely  with  state  funds  and 
is  designed  to  enhance  the  educational  opportunities  available  to  students  from  Iowa's 
schools. 

Second,  a  group  of  Iowa  educational  organizations  prepared  a  proposal  titled  the  Iowa 
Distance  Education  Alliance  (IDEA),  and  in  October  1992  received  notification  of  funding 
from  the  US  Department  of  Education's  Star  Schools  Program.  The  IDEA's  primary  purpose 
is  to  infuse  distance  education  into  the  schools  and  colleges  of  Iowa. 

One  component  of  the  IDEA  is  to  begin  a  comprehensive  research  plan  that  will  investigate 
the  use  of  the  ICN  and  the  activities  of  the  IDEA.  During  1993,  twenty  research  studies 
were  started.  They  are  based  on  Rogers'  (1983)  Diffusion  of  Innovations  Theory,  and  will 
provide  a  foundation  for  additional  studies  to  be  conducted  in  the  future. 

Few  innovations  have  the  potential  impact  that  the  ICN  may  have  on  Iowa  education.  The 
research  plan  of  the  IDEA  will  empirically  examine  the  infusion  of  the  innovation,  "a 
statewide,  full-motion  interactive  fiber  optic  network,"  and  will  provide  information  about 
the  large  scale  adoption  of  distance  education  by  an  entire  state. 

PURPOSE 

This  paper  describes  distance  education  in  Iowa,  the  Iowa  Communications  Network,  The 
Iowa  Distance  Education  Alliance,  and  the  research  plan  underway  in  Iowa.  Research  and 
evaluation  are  central  to  the  implementation  of  distance  education  in  Iowa.  The  process 
used  to  rally  educators  to  conduct  research  is  explained. 


DISTANCE  EDUCATION  IN  IOWA:  BACKGROUND 

Iowa's  approach  to  distance  education  is  based  on  the  belief  thnt  live,  interactive  instruction 
is  fundamental  to  effective  learning.  Interaction  is  made  poHsible  by  the  Iowa 
Communications  Network  (ICN).  The  ICN  is  a  statewide  two-way  full  motion  interactive 
fiber  optic  telecommunications  network.  It  is  designed  to  be  used  by  teuehors  and  students 
in  learning  situations  where  they  not  only  can  see  and  henr  each  other,  but  where  they 
expect  to  see  and  hear  each  other.  Distant  and  local  students  function  togothor  as  a 
learning  group.  They  learn  from  and  with  one  another. 

Key  to  Iowafs  successful  distance  education  system  is  the  concept  of  sharing.  Iowa's  vision 
for  distance  education  is  being  built  around  the  development  of  partnerships  of  schools  that 
share  courses.  For  example,  a  physics  class  originating  in  Jefferson  may  have  students  in 
Sac  City  and  Rockwell  City,  schools  in  two  adjacent  counties.  French  students  in  Sac  City 
have  distant  classmates  in  Jefferson  and  Rockwell  City,  and  a  calculus  class  that  originates 
in  Rockwell  City  is  shared  with  students  in  Sac  City  and  Jefferson.  All  three  schools  provide 
courses  to  partner  schools  and  receive  instruction  from  neighbors.  Classes  are  small  with 
enrollments  of  30-35  or  less,  and  are  taught  by  teachers  prepared  in  the  skills  needed  by 
distance  educators. 


826 


Iowa's  approach  to  distance  education  is  based  on  several  beliefs  about  education.  The 
United  States  has  historically  been  perceived  as  having  the  finest  education  system  ever 
developed.  Local  control,  small  classes,  rapport  between  teachers  and  students,  and  highly 
personalized  instruction  are  hailed  as  important  characteristics  of  this  respected  system. 
On  the  other  hand,  telecommunications-based  education  is  often  perceived  as  the  antithesis 
to  these  attributes.  Distance  education  and  telecommunications  create  the  image  of  a 
centralized  curriculum,  a  single  source  of  information,  and  large  classes  with  little  or  no 
interaction  between  the  teacher  and  students.  Some  feel  that  the  traditional  values  of 
education  and  increased  use  of  technology  are  incompatible  with  one  another.  Iowa 
educators  are  attempting  to  prove  these  critics  are  wrong. 

Of  the  many  projects  of  the  last  few  years  that  have  promoted  the  use  of  technology,  few 
have  been  as  successful  as  the  US  Department  of  Education's  Star  Schools  Program.  The 
Star  Schools  Program  began  in  1987  "to  encourage  improved  instruction  in  mathematics, 
science,  foreign  languages,  literacy  skills,  and  vocational  education  for  underserved 
populations  through  the  use  of  telecommunications  networks."  Many  star  schools  projects 
have  used  communications  satellites  to  deliver  courses  to  large  numbers  of  students  located 
in  dozens  of  cities  and  states.  In  1992,  a  new  approach  to  distance  education  was 
recognized  by  the  Star  Schools  Program  when  a  proposal  submitted  by  an  alliance  of 
educational  organizations  in  the  state  of  Iowa  was  funded. 

Iowa's  project  demonstrates  a  distance  education  system  that  uses  a  statewide  two-way  full 
motion  interactive  fiber  optic  telecommunications  network.  The  "Iowa  Distance  Education 
Alliance:  Partnerships  for  Interactive  Learning  Through  Telecommunications  (IDEA),"  the 
name  of  Iowa's  Star  Schools  project,  demonstrates  that  historically  important  characteristics 
of  an  effective  educational  environment  can  be  combined  with  educational  technology  to  bring 
the  best  of  both  to  the  student  faced  with  the  challenge  of  being  a  citizen  of  the  21st  century. 
The  use  of  fiber  optic  technology,  because  of  its  extensive  capacities  and  flexibility,  provides 
unique  opportunities  for  augmenting  the  instructional  process  beyond  what  is  possible  using 
other  distance  delivery  technologies.  The  IDEA  demonstrates  the  use  of  a  system  that 
emphasizes: 

*  local  control  of  the  curriculum, 

*  active  involvement  by  educators  from  local  school  districts, 

*  interactive  instruction, 

*  statewide  alliances  and  regional  partnerships, 

*  preservice,  inservice,  and  staff  development  activities, 

*  implementation  using  existing  organizations  and  expertise,  and 

*  research-based  instructional  decision  making. 

The  Iowa  Communications  Network  (1CN) 

Ontral  to  the  successful  completion  of  the  Iowa  Distance  Education  Alliance  project  is  the 
Iowa  Communications  Network  (ICN;  Figure  1),  The  ICN  is  a  statewide  two-way  full 
motion  interactive  fiber-optic  telecommunications  network  with  at  least  one  point  of  presence 
in  each  of  Iowa's  99  counties.  The  ICN  links  colleges,  universities  and  secondary  schools 
throughout  the  state  and  was  constructed  entirely  with  state  and  local  funds. 

The  plan  for  the  ICN  was  completed  and  adopted  by  the  Iowa  legislature  in  1987. 
Construction  of  the  fiber  optic  backbone  portion  of  the  network  was  completed  during  1993. 
The  ICN  will  ultimately  connect  hundreds  of  schools,  colleges,  regional  libraries,  and 
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government  ill  agencies.  In  addition  to  the  capability  of  transmitting  up  to  48  simultaneous 
video  channel^  the  ICN  will  carries  data  and  voire  traffic,  ami  as  demand  increases  the 
system  is  easily  expandable  without  the  need  lor  "opening  the  trench11  to  lay  more  Fiber.  The 
IDEA  Star  Schools  initiative  is  developing  and  enhancing  the  human  and  technical  resources 
necessary  to  make  effective  use  of  the  ICN. 


Partnerships  of  Iowa  educational  organizations  are  implementing  the  goals  and  objectives  of 
the  Iowa  Distance  Education  Alliance.  Partnerships  in  Iowa  are  also  referred  to  as  alliances 
because  they  are  a  "joining  for  a  common  purpose."  which  is  the  appropriate  infusion  of  live, 
two-way  interactive  telecommunications  into  the  educational  systems  of  the  state  of  Iowa. 

The  Iowa  Distance  Education  Alliance  was  formed  as  the  result  of  a  collaborative  effort  of 
teachers  and  administrators  from  local  school  districts,  the  Iowa  Department  of  Education, 
Iowa  Public  Television,  Iowa's  community  colleges,  area  education  agencies,  and  public 
(regent)  and  independent  colleges  and  universities,  with  support  from  teacher  and 
administrator  professional  organizations  and  the  state's  K-12  school  boards.  This  alliance  of 
Iowa's  educational  organizations  is  responsible  for  completing  the  IDEA  project. 

Educational  organizations  participating  in  the  Iowa  Distance  Education  Alliance  are 
organized  into  several  components  to  ensure  that  the  activities  of  the  project  are  completed 
(Figure  2).  First,  15  regional  partnerships,  organized  in  accordance  with  Iowa's  merged  area 
education  structure,  were  formed.  The  state's  15  area  community  colleges  and  15  area 
education  agencies  which  share  common  boundaries  are  collaborating  with  teachers  and 
administrators  from  local  schools  to  plan  for  staff  development,  inservice  activities,  and 
course  offerings.  Classrooms  have  been  equipped.  Teachers  have  been  trained.  Curriculum 
materials  ere  under  development.  Staff  in-service  activities  were  offered,  and  beginning 
Fall,  1993,  courses  began  to  be  shared. 

Second,  a  clearinghouse  was  established  by  the  Iowa  Department  of  Education.  The 
clearinghouse  provides  quick  and  comprehensive  access  to  information  about  interactive 
telecommunications,  and  it  coordinates  access  for  Iowa  Distance  Education  Alliance  partners 
to  other  computer  and  telecommunications  networks,  such  as  the  Internet. 

The  third  component  of  the  project  is  the  Teacher  Education  Alliance  which  is  being 
coordinated  by  faculty  from  each  of  the  state's  three  public  universities  working  closely  with 
teachers,  teacher  education  faculty  from  the  independent  colleges  and  universities,  and  the 
area  education  agencies.  The  Teacher  Education  Alliance  has  developed  a  process  for,  and 
materials  being  used  in,  preservice  and  inservice  teacher  education  experiences.  Materials 
are  being  used  to  assist  teachers  in  curriculum  revision  activities  and  to  prepare  them  for 
instruction  of  students  at  a  distance.  Workshops  have  been  held.  Inservice  sessions  have 
been  delivered,  and  curriculum  plans  for  staff  development  are  being  made  available. 
Finally,  the  staff  of  the  Teacher  Education  Alliance  developed  and  are  completing  a 
comprehensive  plan  for  research  and  evaluation  to  determine  the  unique  contributions  of  the 
Iowa  approach  to  the  theory  and  practice  of  distance  education.  Currently,  twenty  research 
studies  are  underway,  and  more  are  being  planned. 

The  fourth  component  of  the  Iowa  Distance  Education  Alliance  is  the  project  management 
structure  which  is  coordinated  by  Iowa  Public  Television  and  the  Iowa  Department  of 
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Education.  The  project  manager's  primary  responsibilities  are  to  administer  the  activities  of 
the  project  and  to  interact  for  the  alliance  with  organizations  in  Iowa  and  elsewhere. 

Goals  and  Activities 

The  projects  goals  are  listed  in  Figure  #3.  To  accomplish  these  goals  a  number  of  at  tivities 
were  undertaken.  Some  of  the  more  significant  activities  were: 

1.  During  1993,  each  of  the  104  endpoints  of  the  ICN  were  connected  to  a  fully  equipped 
distance  education  classroom. 

2.  During  1993,  a  public  awareness  program  w  ;■  planned  and  implemented  that 
familiarized  Iowa  citizens  with  the  concept  of  distance  education,  with  the  capabilities  of  the 
Iowa  Communications  Network,  and  with  the  purposes  of  the  Iowa  Distance  Education 
Alliance. 

3.  The  framework  for  sharing  of  distance  education  courses  was  developed. 

4.  Faculty  from  Iowa's  colleges  and  universities  collaborated  to  develop  a  curriculum 
implementation  plan  that  is  being  used  so  that  graduates  of  Iowa's  teacher  education 
programs  are  prepared  to  become  distance  education  teachers, 

5.  During  the  Spring  and  Summer  of  1993,  hundreds  of  Iowa  teachers  were  prepared  to 
teach  using  the  ICN. 

6.  Faculty  from  Iowa's  universities  and  colleges  collaborated  with  representatives  from  the 
state's  area  education  agencies  to  ensure  that  distance  education  courses  reflected  current 
thinking  and  reform  efforts  in  science,  mathematics,  foreign  language,  vocational  education, 
and  literacy  education. 

7.  a  comprehensive  research  plan  based  on  the  principles  of  diffusion  of  innovations  was 
formulated  and  begun.  This  plan  identified  a  number  of  action -research  projects  that  dealt 
with  local  concerns  in  addition  to  studies  that  investigated  more  fundamental  distance 
education  issues. 

Finally,  the  partners  of  the  Iowa  Distance  Education  Alliance  established  in  Iowa  the 
framework  for  the  infusion  of  the  principles  and  practices  of  distance  education  throughout 
the  state.  The  Iowa  Distance  Education  Alliance  is  a  model  for  other  states  and  regions  that 
plan  the  large  scale  implementation  of  a  distance  education  system  that  is  based  on  the 
concept  of  local  control  of  courses  offered  using  live  two-way  interactive  telecommunications. 


THE  RESEARCH  SITUATION 

The  feelings  in  Iowa  related  to  distance  education  have  traditionally  been  volatile,  and  this 
presents  the  opportunity  for  many  research  studies.  Specifically,  the  situation  in  Iowa  is 
characterized  by  the  following. 

1.  The  ICN  was  planned  for  the  educational  community  of  Iowa  by  state  government,  most 
notably  the  administrative  and  legislative  branches.  No  consensus  of  acceptance  was 
sought  from  or  given  by  Iowa's  educators.  In  other  words,  the  ICN  is  a  top-down  innovation 
that  was  built  and  now  is  available  to  Iowa  educators. 

2.  Only  a  small  percentage  of  Iowa  educators  and  Iowa  citizens  clearly  understand  distance 
education  and  the  ICN,  but  most  are  aware  that  the  state  is  spending  over  $100  million  to 
construct  the  ICN. 

3.  Many  educators  and  citizens  are  skeptical  of  the  concept  of  distance  education,  and 
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opposed  the  construction  of  the  ICN. 

4.  Distance  education  has  gained  acceptance  as  a  discrete  category  of  education. 

5.  Fiber  optics  communication  is  the  most  effective  transmission  method  for  live,  two  way, 
full  motion  interactive  telecommunication,  even  though  it  is  not  the  only  effective 
transmission  method,  or  the  most  cost  effective  transmission  method.  In  Iowa,  fiber  optics 
has  been  mandated  by  the  legislature  as  the  primary  medium  for  distance  education. 

6.  Most  teachers  are  not  fnmiliar  with  the  techniques  of  the  distant  educator,  or  the  needs  of 
the  distant  learner, 

7.  Distance  education  research  is  emerging,  is  largely  anecdotal,  is  not  empirically  based, 
and  iiHiinlly  is  reported  as  individual  studies  with  little  or  no  relationship  to  an  ongoing, 
largo-scale  research  plan. 

H,  Theories  of  distance  education  have  been  proposed  and  are  beginning  to  gain  acceptance, 

U.  lingers'  Diffusion  of  Innovations  Theory,  one  of  several  theories  that  provide  a  foundation 
for  research  in  distance  education,  is  considered  an  appropriate  guide  for  a  large-scale 
research  agenda  such  as  the  diffusion  of  distance  education  in  Iowa. 

The  Research  Plan 

First,  a  group  of  Iowa  educators  was  identified  to  serve  as  a  research  and  evaluation 
advisory  panel  (REAP).  This  group  developed  the  plan  for  soliciting  research  proposals  from 
those  interested  in  investigating  the  distance  education  situation  in  Iowa.  A  Request  for 
Proposals  (RFP)  process  was  used  (Appendix  1).  This  RFP  was  sent  to  over  1000  Iowa 
educators  from  a  cross  section  of  academic  disciplines  and  organizations.  The  RFP  process 
was  conducted  in  four  phases.  After  each  phase,  proposals  were  reviewed  by  the  REAP  who 
provided  authors  of  the  proposals  with  suggestions  for  improving  their  studies.  In  some 
cases  proposals  were  revised  and  resubmitted,  based  on  suggestions  made  by  the  REAP.  At 
the  conclusion  of  the  RFP  process,  twenty  studies  were  selected  for  funding  (Appendix  2), 

Next,  a  comprehensive  review  of  the  distance  education  literature  was  commissioned.  This 
document  (Schlosser  and  Anderson,  1994),  was  published  by  the  Association  for  Educational 
Communications  and  Technology,  and  included  current  information  on  the  theories,  research, 
and  critical  issues  of  distance  education.  This  monograph  was  distributed  to  the  researchers 
whose  proposals  were  selected  for  funding  to  assist  them  in  their  research  efforts.  This 
monograph  also  is  used  to  provide  consistency  among  the  research  studies. 

Late  in  1993,  the  RFP  process  was  concluded.  In  order  to  complete  the  research  agenda  and 
to  fill  gaps  in  coverage,  additional  studies  were  solicited  from  researchers.  Two  examples  of 
new  studies  include  a  Delphi  study  that  will  attempt  to  develop  a  vision  statement  for  how 
distance  education  in  Iowa  should  evolve,  and  a  study  that  will  review  and  summarize  the 
characteristics  of  the  distance  education  classrooms  connected  to  the  ICN,  This  summary 
will  include  a  photograph  of  the  classroom,  a  summary  of  the  classroom's  capabilities,  and  a 
rating  of  the  classroom's  utility  as  a  distant  learning  site.  Finally,  a  synthesis  of  the  most 
effective  features  of  classrooms  will  be  developed  and  a  model  classroom  will  be  proposed. 

Next,  summaries  of  the  funded  research  studies  are  being  published  widely.  This  is  to 
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inform  Iowa  educators  of  the  scope  of  the  project's  research  plan.  As  studies  are  completed 
they  will  be  published  as  monographs  and  distributed  to  those  who  are  interested.  Finally, 
an  Encyclopedia  of  Distance  Education  Research  in  Iowa  will  be  published.  The 
Encyclopedia  will  be  published  late  in  1994  and  will  include  the  entire  research  agenda  of 
the  Iowa  Distance  Education  Alliance  project  including: 

•  the  research,  theory,  and  issues  monograph, 

•  the  final  reports  of  the  project's  research  studies, 

•  a  review  of  the  project's  research  plan,  and 

•  a  summary  of  the  findings  of  the  research  activities  of  the  project. 

CONCLUSION 

"Finally,  the  most  fundamental  and  most  important  characteristic  of  a  profession  is  that  the 
skills  involved  are  founded  upon  a  body  of  intellectual  theory  and  research.  Furthermore, 
this  systematic  theory  is  constantly  being  expanded  by  research  and  thinking  within  the 
profession  .  .  .  the  practice  of  a  profession  cannot  be  disjoined  from  its  theoretical 
understanding  and  vice  versa  .  .  .  The  antithesis  to  a  profession  is  an  avocation  based  upon 
customary  activities  and  modified  by  the  trial  and  error  of  individual  practice.  Such  an 
avocation  is  a  craft. .  .  The  difference  between  the  bricklayer  and  the  architect  lies  right 
here."  (Finn,  1953,  p.  9.) 

In  Iowa,  every  attempt  is  being  made  to  insure  that  the  practice  of  distance  education  is 
based  on  theory  supported  by  research.  The  results  of  the  research  plan  described  above  will 
support  the  professionalization  of  distance  education  in  Iowa,  and  nationwide. 
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Abstract 


Research  that  identifies  factors  that  facilitate  information  processing  and  enhance 
performance  without  reducing  group  confidence  and  decision.satisfaction  may  influence 
future  development  of  groupwork  systems.  The  Cognitive  Closure  Model  of  Decision 
Satisfaction  provides  a  framework  for  research  that  explores  the  relationships  among  need 
for  closure,  extent  of  information  processing,  and  subjective  certainty  (cognitive  closure),  as 
contributing  factors  to  decision  satisfaction  in  group  contexts.  Improvement  of  current 
systems  that  facilitate  group  work  used  in  business  settings  offers  potential  for  the  future 
development  of  such  systems  for  educational  applications. 
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Introduction 


The  importance  placed  on  the  construct  "satisfaction"  in  groups  is  evidenced  by  the 
amount  and  variety  of  contexts  in  which  it  has  been  studied  and  measured.  For  example, 
Maier  (1970)  cited  the  importance  of  member  satisfaction  for  decision  adoption  and 
implementation.  Van  de  Ven  and  Delbecq  (1974)  operationalized  group  effectiveness  to 
include  performance  and  satisfaction.  Keller  (e.g.  1983)  specifies  satisfaction  as  one  of  four 
critical  factors  in  his  model  for  designing  motivating  instruction.  Yet,  despite  its 
acknowledged  importance  the  construct  is  not  well  understood. 

Although  their  goals  and  contexts  may  be  quite  different,  a  close  look  at  satisfaction 
in  the  group  decision -making  literature  may  provide  useful  insight  into  the  design  of 
computer-based  systems  that  support  group  learning  and  problem-solving  tasks.  The 
research  on  and  development  of  computer-based  group  decision  support  systems  for  use  in 
business  settings  may  have  promising  applications  in  educational  settings. 

This  paper  reviews  the  literature  on  cognitive  and  motivational  issues  in  both  group 
decision-making  and  learning  contexts,  due  to  their  dual  focus  on  task  and  process 
satisfaction,  and  examines  the  concept  of  closure  as  a  possible  contributor  to  group 
member  satisfaction.  This  is  followed  by  a  presentation  and  description  of  The  Cognitive 
Closure  Model  of  Decision  Satisfaction  (Venkatesh,  Small  &  Verville,  1993)  and  concludes 
with  a  brief  discussion  of  potential  applications  to  education. 

Group  Member  Satisfaction 

In  learning  situations,  satisfaction  is  often  described  as  the  sense  of  accomplishment 
that  learners  feel  at  the  conclusion  of  a  learning  event  when  outcomes  of  their  efforts  are 
consistent  with  their  expectations  (e.g.  Keller,  1983).  Keller  maintains  that  instruction 
may  be  designed  to  help  people  feel  good  about  their  learning  accomplishments,  thereby 
resulting  in  learning  satisfaction.  This  suggests  that  learning  satisfaction  occurs  at  or  near 
the  end  of  either  each  learning  chunk  or  the  total  learning  event. 

Assessment  of  learning  satisfaction  often  focuses  on  the  individual's  satisfaction 
with  his  or  her  learning  accomplishments.  McCombs  (1982)  and  Keller  (1983)  mention 
intrinsic  factors  such  as  personal  control  and  responsibility  as  contributors  to  learner 
satisfaction.  Small  and  Gluck  (in  press)  found  that  adults  perceive  external 
reinforcements  such  as  feedback,  encouragement,  expectations,  praise,  and  reward  as 
factors  closely  related  to  satisfaction.  Manteuffel  (1982)  cited  both  control  and  reward  as 
major  characteristics  for  a  satisfied  adult  learner. 

Frequently,  satisfaction  is  tied  to  the  completion  of  a  learning  task  by  an  individual 
in  a  learning  situation.  However,  often  learners  are  required  to  work  in  groups,  use  group 
processes,  accomplish  group  tasks,  and  be  assessed  according  to  group  outcomes.  In  these 
situations,  group  member  satisfaction  may  not  reflect  individual  satisfaction;  i.e.  an 
individual  could  be  satisfied  that  the  group  reached  a  group-level  goal  but  lack  personal 
closure  on  the  efficacy  of  the  result  or  the  process. 

Hecht  (1978)  has  stated  the  need  for  theoretic  work  in  the  measurement  of 
satisfaction.  Exploration  of  the  factors  that  influence  satisfaction  in  group  decision-making 
situations  has  led  researchers  to  distinguish  between  task-related  and  process-related 
components  of  satisfaction.  In  relation  to. task-related  factors,  several  researchers  have 
reported  a  link  between  group  task  accomplishment  and  group  member  satisfaction. 
Heslin  and  Dunphy  (1960)  reported  that  groups  scoring  low  on  perceived  task 
accomplishment  tended  to  report  low  group  member  satisfaction,  while  Marquis, 
Guetzkow  and  Heyns  (1951)  found  satisfaction  to  be  higher  in  groups  that  reported  a  high 
degree  of  accomplishment.  They  determined  that  groups  that  completed  a  larger 
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percentage  of  their  agenda  were  more  satisfied  than  groups  that  did  not. 

Other  studies  have  identified  process-related  factors  of  task  accomplishment  as 
predictors  of  group  member  satisfaction.  Hrycenko  and  Minton  (1974)  suggest  that 
member  satisfaction  with  the  task-performance  procedure  chosen  may  contribute  to  overall 
satisfaction.  The  process  dimension  is  implicit  in  Collins  and  Guetzkow's  (1964) 
observation  that  perception  of  movement  toward  the  task  goal  may  be  positively  related  to 
satisfaction.  Preparing  and  adhering  to  the  meeting  agenda  and  keeping  the  problem  in 
focus  during  the  meeting  have  also  been  cited  as  factors  promoting  satisfaction  (e.g. 
Kriesberg,  1950). 

Closure 

In  the  education  literature,  closure  is  often  presented  as  a  norm;  i.e.  that  a  learning 
task  must  be  brought  to  closure  (e.g.  Phillips,  1987;  Dubelle,  1986).  McMillin  and 
Newman(1991)  cited  closure  as  one  of  six  important  elements  for  effective  instruction. 

Closure  is  often  associated  with  the  idea  of  "completeness,"  as,  for  example,  in 
Gestalt  theory  which  proposes  that  one  responds  to  a  situation  as  a  complete  and 
unanalysable  whole  rather  than  a  sum  or  specific  elements.  Dubelle  (1986)  describes 
closure  as  the  outcomes  of  an  activity  that  brings  the  major  points  of  a  lesson  into  focus  so 
they  may  be  perceived  as  an  organized  whole  and  as  the  individual's  need  to  smooth  or 
complete  what  is  unfinished.  Stacey  and  DeMartino  (1963)  emphasize  the  importance  of 
closure  as  striving  for  some  form  of  completion  of  an  activity. 

Reigeluth  (1984)  prescribes  a  "zooming  in  and  out"  approach  to  instruction  that 
allows  students  to  study  specific  content  or  skills  but  always  in  the  context  of  the  larger 
body  of  knowledge  in  which  they  fit.  Similarly,  Keller  (1984)  emphasizes  the  need  for 
learners  to  perceive  various  pieces  of  content  as  fitting  into  a  whole,  thereby  experiencing 
closure  and  a  sense  of  accomplishment.  Brophy  (1987)  contends  that  students  experience  a 
sense  of  accomplishment  when  they  complete  meaningful  tasks.  Wlodkowski  (1991) 
advocates  attaining  closure  when  insructing  adult  learners,  stating  that  it  "enhances 
learner  motivation  because  it  affirms  the  entire  process,  reinforces  the  value  of  the 
experience,  directly  or  indirectly  acknowledges  competence,  increases  cohesivencss  within 
the  group,  and  encourages  the  surfacing  of  inspiration  and  other  beneficial  emotions  within 
the  learners  themselves  (pp  247-248).  Therefore,  closure  may  relate  to  subjective  certainty 
or  confidence  related  to  the  result  (cognitive  closure)  or  related  to  the  completion  of  the 
process  (process  closure). 

Cognitive  closure.  The  motivation  to  attain  cognitive  closure  has  been  examined  in 
regard  to  subjective  certainty;  that  is  the  need  to  closure  may  motivate  people  to  prefer 
certainty  and  coherence  over  indeterminacy.  Kruglanski  (e.g.  1989)  has  examined  the 
need  for  closure  within  a  motivational  theory  of  cognition.  Cognitive  closure  occurs  when  a 
definite  answer  to  a  question  is  obtained,  leaving  no  ambiguity  or  confusion  (Kruglanski 
and  Freund,  1983).  Need  for  closure  represents  a  need  to  attain  assured  knowledge  that 
"affords  predictability  and  a  base  for  action"  (Kruglanski,  1989,  p.  14). 

In  a  group  context,  the  need  for  closure  may  be  reflected  in  the  group's  motivation  to 
collectively  develop  a  genuine  "social  reality"  (Festinger,  1950)  or  consistent  problem 
representation.  Such  a  motive  may  prompt  the  group  to  resolve  differences  so  that 
authentic  agreement  may  result.  This  drive  for  group  consensus  may  be  similar  to  an 
"individual's  need  for  personal  closure  in  his  or  her  own  cognitive  system,  that  is,  for 
intrapcrsonal  consistency  among  the  individual's  cognitions  giving  rise  to  a  sense  of 
coherent  knowledge  or  subjective  reality"  (Kruglanski  and  Webster,  1991,  p.  223), 

Process  closure.  Both  activity  predictability  and  completion  of  activity  sequences  may 
facilitate  closure  (Maier,  1970).  Group  task  strategies  facilitate  both  coordination  of  group 
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effort  and  the  determination  of  task  progress  and  may  help  members  generate 
expectations  about  when  and  how  these  activity  sequences  will  conclude  (Losada  et  al., 
1990).  In  educational  situations,  learners  engaged  in  a  learning  task  strive  for  closure  to 
an  activity  under  way  (Stacey  and  DeMartino,  1963).  When  closure  is  not  achieved, 
learners  may  feel  "left  up  in  the  air"  and  psychologically  unsettled  (Cutietta,  1984). 

Satisfaction  and  Closure 

The  literature  reviewed  to  date  does  not  adequately  address  the  possible 
relationship  between  satisfaction  and  closure.  In  the  education  literature,  closure  is 
commonly  presented  as  a  desired  end  state;  i.e.  that  the  learning  task  should  be  brought 
to  closure  for  learning  satisfaction  to  occur  and  that  it  may  require  more  than  one  learning 
session  to  accomplish  closure. 

In  the  group  literature,  Hagen  and  Burch  (1985)  found  that  both  perception  of 
closure  on  a  group  task  and  attainment  of  closure  for  task  direction  promoted  satisfaction. 
However,  the  factors  behind  closure  were  not  adequately  explored,  nor  was  the  concept  of 
closure  clearly  defined.  Furthermore,  the  psychometric  properties  of  the  instrument  used  to 
measure  satisfaction  were  not  adequately  described. 

Van  de  Ven  and  Delbecq  (1974)  compared  groups  using  interacting,  delphi,  and 
nominal  group  technique  (NGT).  They  found  the  NGT  groups  attained  high  closure  while 
delphi  and  interacting  groups  members  attained  either  low  closure  or  "closure  with 
detachment"  (p.  619)  but  closure  was  implicitly  (not  explicitly)  defined  as  task 
accomplishment.  Furthermore,  NGT  groups  also  reported  being  significantly  more  satisfied 
than  the  other  groups.  Here,  satisfaction  was  operationalized  to  include  perceptions  of 
both  process  and  quality  of  the  group's  performance.  They  enumerate  "facilitative 
characteristics"  of  the  NGT:  structured  meeting  process,,  balance  bewteen  task  and 
maintenance  focus,  problem-centerednes3,  opportunity  for  members  to  "think  through  and 
write  down  ideas"  and  so  on  (Van  de  Ven  &  Delbecq,  1974,  pp  617-619).  Although  this 
result  hints  at  a  linkage  between  satisfaction  and  closure,  Van  de  Ven  &  Delbecq  do  not 
explicitly  make  this  connection.  They  also  do  not  specify  whether  these  facilitative 
characteristics  are  examples  of  more  general  underlying  mechanisms  that  promote 
cognitive  closure  and,  if  so,  what  those  mechanisms  are.  Therefore,  while  prior  group 
research  has  addressed  closure  as  an  outcome  measure,  the  construct  is  seldom  explicitly 
defined  and  the  components  of  closure  remain  unidentified. 

Closure  and  Confidence 

In  group  decision-making  situations,  satisfaction  may  be  considered  an  affective 
response  of  group  members  toward  the  decision  made,  while  confidence  is  a  cognitive 
response.  Wlodkowski  (1991)  asserts  that  confidence  occurs  once  a  person  knows  with 
some  degree  of  certainty  that  he  or  she  is  capable  or  adept  at  what  was  learned. 

While  satisfaction  and  confidence  are  not  the  same  (Sniezek,  1980),  there  is  evidence 
that  they  are  highly  related  constructs,  Keller  (1983)  identifies  four  critical  factors  for 
motivation  and  describes  them  in  relationship  to  expectancy-value  theory,  with  both 
confidence  and  satisfaction  linked  to  expectancy  for  success  or  failure.  According  to  Keller, 
confidence  is  built  on  such  factors  as  awareness  of  expectations,  personal  control,  and  a 
history  of  success  while  satisfaction  depends  on  meaningful  applications,  positive 
outcomes,  and  consistency.  In  a  study  that  explored  the  relationships  of  Keller's  four 
factors  and  to  identify  the  effective  instructional  strategies  that  are  most  closely  related  to 
each  of  them,  Small  and  Gluck  (in  press)  found  a  significant  relationship  between 
confidence  and  satisfaction. 


HIM 


3  f 


Kruglanski  (1989)  notes  that  the  need  for  closure  represents  a  quest  for  assured 
knowledge.  In  this  paper,  closure  is  defined  as  a  state  characterized  by  confidence  or 
subjective  certainty  (cognitive  closure),  terms  often  used  synonymously  (Sniezek,  1980). 
This  relates  to  the  characterization  of  closure  as  task  accomplishment  (Hagen  &  Burch, 
1985;  Van  de  Von  &  Delbecq,  1974)  because,  in  decision  making  situations,  a  task  is 
perceived  as  "done"  when  subjective  certainty  about  the  decision  is  deemed  acceptable  by 
the  decision-maker  (e.g.  Corhin,  1980). 

In  their  research,  Kruglanski  and  his  associates  manipulated  the  need  for  closure  as 
an  independent  variable.  We  examine  closure  (subjective  certainty)  as  a  dependent 
variable.  Our  work  focuses  on  the  effects  of  information  processing  on  closure  and  whether 
attaining  cognitive  closure  promotes  decision  satisfaction.  The  Cognitive  Closure  Model  of 
Decision  Satisfaction  (see  Figure  1)  is  based  on  the  literature  and  our  previous  work  in 
this  area. 


Place  Figure  1  about  here 


The  Cognitive  Closure  Model  of  Decision  Satisfaction 

The  Cognitive  Closure  Model  of  Decision  Satisfaction  begins  with  the  proposition 
that  need  for  closure  affects  the  extent  of  information  processing  (Kruglanski,  1989)  and 
explores  the  implications  of  this  proposition  for  closure  (as  a  dependent  variable)  and 
decision  satisfaction.  The  following  sections  of  this  paper  describe  this  model  through  a 
discussion  of  each  pair  of  posited  relationships  leading  to  final  satisfaction;  i.e.  need  for 
closure  and  extent  of  information  processing;  extent  of  information  processing  and  cognitive 
closure  (defined  as  confidence  or  subjective  certainty),  and  cognitive  closure  and  decision 
satisfaction. 

Need  for  Closure  and  Extent  of  Information  Processing 

Knowledge  is  assumed  to  consist  of  propositions  in  which  a  person  has  a  given 
degree  of  confidence  (Kruglanski,  1989).  As  such,  knowledge  has  two  components-- 
hypothesis  generation  during  which  propositions  are  generated  and  hypothesis  validation  in 
which  a  given  degree  of  confidence  is  attached  to  the  hypotheses.  Knowledge  acquisition, 
involving  epistemic  activity,  thus  entails  information  processing. 

The  hypothesis  generation-validation  model  might  be  considered  a  general 
characterization  of  the  process  by  which  knowledge  is  acquired.  In  a  study  that  examined 
the  impact  of  telecommunication  system  design   id  instructor  style  factors  on  student 
perceptions  of  learning  and  satisfaction,  Walker  and  Hackman  (1991)  found  that  the 
amount  of  information  received  was  the  single  greatest  contributor.  However,  Hooper 
(1992)  suggeats  that  cognitive  benefits  associated  with  cooperative  learning  situations  are 
more  closely  related  to  giving  than  to  receiving  information.  He  further  states  that  M(t)he 
processes  associated  with  generating  explanations  appear  to  force  students  to  process 
information  deeper  than  the  processes  association  with  listening  to  explanations  of  lesson 
content"  (p.27). 

Kruglanski  (1989)  differentiates  between  two  "epistemically-relevant  motivations" 
that  affect  the  extent  of  information  processing  (both  breadth  and  depth)  in  knowledge 
acquisition;  i.e.  high  and  low  need  for  closure.  Information  processing  may  be  less 
extensive  (breadth)  under  high  need  for  closure  conditions  than  low  need  for  closure 
conditions  (Mayselcss  &  Kruglanski,  1987)  while  it  may  be  more  intensive  (in-depth) 
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under  certain  conditions.  That  is,  low  need  for  closure  subjects  were  more  sensitive  to 
alternative  hypotheses,  considered  information  that  was  inconsistent  with  prior  opinions, 
and  were  more  open  to  both  global  and  specific  information  about  the  task  than  were  high 
need  for  closure  subjects  (Kruglanski,  1989). 

The  cognitive  motivation  to  attain  firm  knowledge  may  therefore  be  said  to  influence 
the  extent  of  information  processing.  This  model,  however,  does  not  address  two  important 
issues— how  the  process  is  regulated  and  how  the  knowledge  that  is  acquired  through  the 
process  is  judged  to  be  more  or  less  firm.  These  issues  are  considered  critical  because  the 
motivation  to  attain  or  delay  cognitive  closure  entails,  by  definition,  a  characterization  of 
the  decision- maker  as  an  active  evaluator  of  the  adequacy  of  the  information  processing.  It 
also  assumes  the  existence  of  a  standard  of  appraisal  against  which  knowledge  is  judged 
to  be  more  or  less  firm. 

Extent  of  Information  Processing  and  Cognitive  Closure 

The  model  further  posits  that  the  decision -maker  actively  assesses  the  adequacy  of 
information  processing  with  reference  to  the  need  for  closure  (high  vs.  low)  salient  in  the 
situation.  That  is,  the  need  for  closure  may  regulate  the  extent  of  information  processing 
(including  both  hypothesis  generation  and  validation)  before  a  firm  judgment  is  reached 
(Mayseless  &  Kruglanski,  1987;  Kruglanski  &  Freund,  1983), 

In  general,  it  appears  that  high  need  for  closure  (induced  by  either  reducing  the  cost 
to  the  subject  of  judgmental  invalidity  or  by  increasing  the  benefit  of  rendering  an 
expeditious  judgment)  retards  information  processing.  Conversely,  low  need  for  closure 
(induced  by  increasing  the  cost  to  the  subject  of  judgmental  invalidity)  fosters  more 
extensive  information  processing.  Under  either  of  these  conditions,  the  decision-maker 
actively  assesses  the  costs  versus  the  benefits  of  increased  information  processing  in 
conditions  characterized  by  differing  motivations  to  reach  a  firm  decision.  As  a 
consequence,  processing  is  judged  adequate  when  such  a  motivation  is  high  (high  need  for 
closure)  and  inadequate  when  the  motivation  is  low  (low  need  for  closure).  However, 
Kruglanski's  work  does  not  clarify  how  the  decision-maker  determines  that  the  knowledge 
he  or  she  possesses  is  more  or  less  firm  before  deciding  to  decide;  i.e.  by  what  criterion  or 
standard  of  appraisal  the  decision-maker  makes  this  determination. 

It  is  likely  that  cognitive  closure  (defined  here  as  the  minimum  level  of  subjective 
certainty  or  confidence  acceptable  to  the  decision- maker)  will  function  as  such  a  criterion. 
Information  gathering  and  deliberation  are  geared  in  part  to  reducing  the  subjective 
uncertainty  inherent  in  a  decision  to  a  "comfortable"  level  (Cox  &  Rich,  1964).  This  implies 
that  the  decision-maker  may  not  feel  the  need  to  decide  until  this  level  is  reached. 

Consistent  with  this  line  of  reasoning,  Corbin  (1980)  uses  the  "uncertainty  cutoff 
idea  to  discuss  the  timing  of  choice.  This  cutoff  point  is  subjectively  determined  according 
to  an  acceptable  level  of  certainty.  When  subjective  certainty  fails  to  reach  that  level,  the 
decision-maker  may  choose  to  delay  making  a  decision  or  avoid  making  a  decision  entirely. 
Intensive  processing  of  available  information  or  gathering  more  information  may  provide 
an  alternative  means  "for  decreasing  uncertainty  and  for  inducing  the  readincHH  to  decide" 
(Corbin,  1980,  p.  54). 

In  an  attempt  to  define  what  constitutes  an  "acceptable  level"  of  subjective  certainty 
and  how  to  determine  the  "uncertainty  cutoff," 

we  propose  that  the  cutoff  level  may  be  a  function  of  the  need  for  closure.  In  high  need  for 
closure  situations,  the  acceptable  confidence  level  may  be  set  relatively  low  if  the  cost  of 
judgmental  error  is  not  perceived  to  be  great  and  the  reward  is  perceived  to  be  high. 
•  However,  in  low  need  for  closure  situations,  the  decision-maker  may  feel  a  need  to  attain  a 
relatively  high  level  of  confidence  before  making  a  decision  if  the  cost  of  judgmental 
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invalidity  is  perceived  us  higher  than  the  reward  for  possessing  firm  knowledge. 

The  model  asserts  that  the  decision  maker  assesses  both  the  adequacy  of  the  extent 
of  information  processing  and  the  subjective  confidence  level  during  the  decision  making 
process.  The  previous  discussion  indicates  that  a  determination  of  what  is  an  adequate 
level  of  information  processing  may  be  influenced  by  cognitive  and  motivational  factors. 
Decision  makers  may  simply  process  more  information  in  order  to  increase  the  subjective 
confidence  level,  terminating  such  processing  when  the  confidence  level  exceeds  the 
uncertainty  cutoff.  Decision  makers  may  decide  to  halt  processing  and  make  a  decision  if 
they  believe  that  they  had  put  forth  "enough"  effort  on  the  task  and  therefore  believe  the 
probability  of  being  correct  is  high,  resulting  in  high  final  confidence.  Thus  the  model 
explicity  indicates  a  reciprocal  relationship  between  subjective  confidence  and  information 
processing  (see  Figure  1);  i.e.  information  processing  helps  increase  confidence  in  general, 
although  under  certain  circumstances  increased  processing  may  reduce  confidence  (Sniezek, 
1991).  Information  processing  would  continue  until  subjective  certainty  is  above  the 
acceptable  level  of  confidence  (or  reaches  the  uncertainty  cutoff)  which  defines  the  moment 
of  choice. 

Information  overload  may  negatively  affect  decision-makers'  ability  to 
"integrate  and  cope  with  all  that  information. ..lead(ing)  to  dissatisfaction"  (Cummings, 
O'Connell  and  Huber,  1976,  p.  234).  In  a  study  exploring  the  ways  people  approach  the 
task  of  gathering  information  on  a  specific  topic  from  a  multimedia  information  source, 
Small  and  Ferreira  (1994)  found  that  some  subjects  expressed  frustration  and  uncertainty 
about  the  amount  of  information  available  for  access  when  using  a  multimedia 
information  system.  This  phenomenon  may  have  contributed  to  a  lack  of  subjective 
certainty  about  the  adequacy  of  the  information  search  process  or  the  information 
accessed,  resulting  in  a  perceived  lack  of  closure. 

Cognitive  Closure  and  Decision  Satisfaction 

Although  cognitive  closure  may  load  to  decision  Hatisfaction,  there  is  u  need  for 
research  that  investigates  the  effects  of  different  levels  of  need  for  closure  and  extent  of 
information  processing  on  both  factors,  For  example,  when  the  need  for  closure  is  valued 
and  attained  (high),  is  decision  satisfaction  assured?  If  the  need  for  closure  is  low  (not 
immediately  desired)  but  external  constraints  (e.g.  time)  force  a  decision,  does  this  lead  to 
lowered  or  lack  of  confidence  and  decision  satisfaction?  When  each  group  member 
possesses  different  information,  does  the  perceived  necessity  to  share  information  influence 
interaction  and  the  extent  of  information  processing?  When  all  group  members  possess  the 
same  information,  does  interaction  decrease  and  is  the  extent  of  information  processing 
affected?  Our  current  research  is  exploring  some  of  these  issues. 

Conclusions 

Group  decision  support  systems  (GDSS)  (e.g.  GroupSystems,  VisionQuest)  are  a  set 
of  networked  electronic  tools  that  facilitate  group  work,  most  often  in  business  settings.. 
They  typically  include  software  that  allows  participants  to  conduct  group  activities,  such 
as  anonymous  electronic  brainstorming  and  organization  and  ranking  of  alternatives. 

Research  that  identifies  factors  that  facilitate  information  processing  and  enhance 
performance  without  reducing  group  confidence  and  decision  satisfaction  may  influence 
future  development  of  groupwork  systems.  The  Cognitive  Closure  Model  of  Decision 
Satisfaction  provides  a  framework  for  research  that  explores  the  relationships  among  need 
for  closure,  extent  of  information  processing,  subjective  certainty,  and  decision  satisfaction. 

Although,  the  current  model  does  not  incorporate  a  feedback  component,  Small  and 
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Gluck  (in  press)  identified  feedback  as  a  unique  term  related  to  satisfaction.  Hooper 
(1992)  stated  the  need  "to  investigate  the  effects  of  feedback  timing  and  the  nature  of 
feedback  in  group  environments"  (p.  36).  Adding  appropriate  feedback  loops  to  the  model 
may  increase  the  likelihood  of  confidence  and  satisfaction  in  group  work.  This  is  an  area 
for  future  research. 

Steeb  and  Johnston  (1981)  suggest  that  visual  aid  organizers  may  help  decision- 
makers deal  with  information  overload  more  effectively. 

This  type  of  intervention  tool  during  the  hypothesis  generation  and  validation  stage  might 
promote  confidence  and  satisfaction.  However,  application  to  one  or  the  other  (generation 
or  validation)  might  decrease  confidence  and/or  satisfaction. 

Research  on  and  development  of  effective  support  systems  for  groupwork  in  business 
settings  holds  promise  for  applications  to  education.  Aiken  (1992)  recommends  the  use  of 
GDSS  technology  in  a  wide  variety  of  educational  situations  (e.g.  foreign  language 
instruction,  classes  for  the  deaf,  group  meetings)  as  an  effective  tool  ui  group  instruction. 
Adaptation  of  these  tools  to  educational  contexts  could  enhance  problem-solving  learning 
activities.  For  example,  such  systems  would  allow  discussions  of  sensitive  or  controversial 
topics  (e.g.  as  drug  abuse,  national  health  care,  the  environment,  conflict  resolution), 
allowing  all  students  to  participate  actively  while  allowing  their  thoughts  and  ideas  to 
remain  anonymous.  Exploration  of  applications  of  GDSS  technology  in  education,  as  well 
as  ways  to  improve  current  group  work  systems,  are  needed. 
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ABSTRACT 


In  educational  and  training  environments  touchers  assign  their  students 
independent  research  tasks  that  require  locating,  feathering,  synthesizing,  and  summarizing 
information  from  one  or  more  information  resources  containing  large,  and  sometimes 
overwhelming,  amounts  of  information.  Often  these  tasks  are  poorly-defined,  requiring 
learners  to  utilize  resources  that  have  not  been  tailored  to  meet  their  specific  information 
needs,  This  may  result  in  information  overload  and/or  information  anxiety.  This  study 
explored  the  information  location  anil  use  activities,  motivation  and  learning  patterns  of 
children  and  adults  using  either  a  print  or  multimedia  resource  to  complete  a  research  task. 
Results  indicated  some  differences  in  engagement  of  text  and  nontext  information, 
expectancy  of  success,  and  type  of  knowledge  representation. 
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INTRODUCTION 

Although  there  have  been  numerous  studies  comparing  learning  and  performance 
with  various  instructional  delivery  systems,  their  results  have  been  largely  mixed  (e.g. 
Clark,  1983).  In  addition,  many  of  these  studies  center  on  a  medium  as  an  instructional 
rather  than  informational  intervention  to  satisfy  a  learning  need.  Furthermore,  research 
questions  focus  on  whether  one  medium  is  better  (i.e.  results  in  higher  learning  and/or 
motivation  outcomes)  than  another  medium,  rather  than  examining  how  learning  occurs 
with  a  specific  information  resource. 

In  educational  and  training  environments  students  may  be  assigned  independent 
research  tasks  that  require  them  to  locate,  gather,  synthesize,  and  summarize  information 
from  one  or  more  information  resources  containing  large,  and  sometimes  overwhelming, 
amounts  of  information.  Often  these  tasks  are  poorly-defined,  requiring  learners  to  utilize 
resources  that  have  not  been  tailored  to  meet  their  specific  information  needs.  Furthermore, 
students,  even  as  young  as  elementary  and  middle  school  age,  are  often  expected  to 
accomplish  the  major  part  of  a  research  task  independently  (Kuhlthau,  1989).  In  addition, 
they  may  use  multimedia  information  resources  that  require  the  processing  of  both  print  and 
non-print  information  simultaneously. 

These  issues  become  magnified  as  the  use  of  information  technologies  (both  stand- 
alone and  distributed)  allow  students  to  gain  access  to  information  in  contexts  that  are 
outside  of  traditional  structured  educational  institutions  (e.g.  homes).  This  may  result  in 
information  overload  and  "information  anxiety,"  described  by  Wurman  (1989)  as  a"  black 
hole  between  data  and  knowledge11  (p.  34).  The  purpose  of  this  study  was  to  explore  the 
attitudes  and  processes  that  students  activate  when  using  different  types  of  information 
resources  to  complete  an  independent  research  task.  The  study  focused' on  three  areas- 
information  location  and  use,  motivation,  and  learning. 

Information  Location  and  Use 

Traditionally,  students  have  conducted  information  retrieval  activities  by  locating 
relevant  print  resources  (e.g,  encyclopedias,  reference  books),  searching  through  them  one  at 
a  time,  extracting  bits  of  relevant  information  from  each  resource,  Once  they  have  extracted 
what  they  perceive  as  important  information  within  one  resource,  they  proceed  to  the  next 
resource  and  repeat  the  process.  Eisenberg  and  Berkowitz  (1990)  describe  this  process  in 
the  context  of  a  larger  information  problem  solving  process.  In  this  study  we  focused  on  three 
information  location  and  use  tasks  consist  of  finding  desired  information  within  a  resource 
either  through  using  an  organizational  tool  (e.g.  index)  or  browsing  (random  or  sequential), 
engaging  the  information  by  viewing,  reading  and/or  listening,  and  extracting  information 
within  a  resource  through  notetaking.  Marchionini  (1989)  likens  these  activities  to 
conducting  the  look  up,  examining  the  results,  and  extracting  relevant  information. 

Current  technologies  that  combine  computer-based  control  and  a  CD-ROM  or 
laserdisk  storage  device  provide  students  with  simultaneous  access  to  integrated 
information  resources  that  utilize  a  variety  of  media  (Grabowski  &  Curtis,  1991). 
Theoretically,  these  multimedia  information  resources  provide  students  with  faster  and 
easier  access  to  potentially  richer  bodies  of  information  than  any  single  medium  resource.  As 
a  result,  information  skills  such  as  finding,  selecting,  and  extracting  become  even  more 
critical  (Small  &  Ferreira,  in  press). 

Several  studies  have  investigated  information  location  and  use  with  computer-based 
and  print  resources.  Mynatt  et  al.  (1992)  found  that  subjects  using  a  hypertext,  nonlinear 
book,  even  with  minimal  training,  are  as  successful  at  an  information  seeking  task  as  those 
using  a  conventional  book.  Shneiderman  (1987)  reported  about  equal  performance  on  an 
information  use  task  with  a  hypertext  and  print  version.  MacKnight  and  Baroni  (1993) 
found  that  students  using  either  print  or  electronic  versions  of  a  manual  to  complete  a 
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research  task  did  not  differ  in  their  ability  to  recognize  and  extract  important  information 
from  either  source  or  in  the  number  of  times  they  bypassed  or  overlooked  important 
information.  The  current  study  investigates  differences  in  information  location  and  use 
activities  (finding,  engaging,  and  extracting  by  children  and  adults  using  a  print-based  or 
multimedia  resource  to  complete  a  research  task. 

Motivation 

The  attitudes  or  motives  users  harbor  in  regard  to  a  particular  information  resource 
may  influence  their  use  of  that  resource  which,  in  turn,  could  influence  their  perceptions 
about  the  accessibility  of  relevant  information  within  that  resource  (Culnan,  1985).  In  this 
study,  motivation  is  defined  broadly  in  terms  of  the  value  and  expectations  of  successful  use 
of  a  resource,  a  theory  known  as  expectancy-value  theory  (e.g.  Porter  &  Lawler,  1968).  Value 
refers  to  the  perceived  importance  of  or  interest  in  the  information  resource,  while  expectancy 
refers  to  the  perceived  accessibility  of  needed  information  and  successful  completion  of  the 
task. 

Taylor  (1986)  identities  a  number  of  characteristics  of  an  information  system  (e.g. 
accessibility,  accuracy,  currency,  reliability,  ease-of-use)  that  add  value  for  the  user  of  that 
system.  For  example,  the  most  effectively  and  efficiently  designed  encyclopedia  would  be 
considered  a  poor  learning  resource  if  it  contained  inaccurate,  outdated,  or  irrelevant 
information  for  meeting  a  learner's  specific  information  needs. 

Previous  research  indicates  that  computer-based  systems  that  stimulate  curiosity 
and  interest  and  allow  greater  user  control  may  result  in  more  positive  attitudes,  greater 
engagement  in,  and  less  anxiety  toward  using  these  systems  (Kinzie,  1990).  Heroman 
(1990)  found  high  student  engagement  to  have  a  positive  relationship  with  satisfaction  with 
an  overall  learning  situation.  Stimulation  deficit  (information  underload)  or  excessive 
stimulation  (information  overload)  often  result  in  boredom  (Klapp,  1986).  The  learner's 
perception  of  his/her  competence  in  successfully  navigating  this  type  of  system  and  retrieving 
all  of  the  relevant  information  may  contribute  to  a  positive  motivation  toward  the  system 
and  successful  accomplishment  of  learning  goals  (Kinzie,  1990).  Keller  (e.g.  1983)  describes 
instructional  motivation  in  terms  of  stimulating  interest  and  curiosity  and  providing 
relevance,  importance,  and  utility  (value-related  strategies),  building  confidence  and 
competence,  and  promoting  satisfaction  through  effort-success  linkages  (expectancy-related 
strategies). 

In  a  study  comparing  computer-based  to  print-based  instruction,  Yang  (1991-2) 
found  that  the  computer-based  instructional  group  showed  significantly  higher  motivation  on 
a  post-treatment  motivation  survey  than  did  the  print-based  instruction  group  and 
concluded  that  the  computer  had  a  strong  motivational  impact  on  students'  learning, 
possibly  resulting  from  the  novelty  and  higher  potential  learner  control  of  that  medium. 

Much  of  the  current  research  has  determined  that  students  who  use  computer-based 
instructional  resources  are  more  nighly  motivated  than  those  using  print  resources.  The 
current  study  compared  changes  in  motivation  levels  for  subjects  required  to  use  a  specific 
resource  to  complete  a  poorly -defined,  independent  research  task  where  (1)  the  resource  is 
learner-contrc  Med  rather  than  instructor-controlled  and  (2)  the  resource  is  informational 
rather  than  instructional  (Grabowski  &  Curtis,  1991). 

Learning  Patterns 

Studies  comparing  learning  with  print  and  non-print  sources,  often  use  traditional 
methods  of  evaluation  such  as  multiple  choice  tests.  These  are  appropriate  when  the  body  of 
content  to  be  learned  is  tightly  controlled,  as  in  a  textbook  or  computer-based  instructional 
program.  However,  when  the  task  is  one  in  which  the  learner  uses  one  or  more  resources 
which  are  informational  rather  than  instructional  in  nature,  the  body  of  content  is  often  not 
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easily  controlled  and  there  is  a  lack  of  learning  guidance. 

Users  of  multimedia  resources  may  independently  select,  process,  and  interrelate 
new  pieces  of  information  with  information  they  already  know  to  construct  new  knowledge 
(DeCorte  1990).  They  enjoy  a  new  sense  of  empowerment  by  having  control  over  format, 
type  and  amount  of  information,  pace,  and  sequence  of  content  (Small  &  Grabowski,  1992). 

Regardless  of  the  resource,  students  seek  to  find  an  order,  structure  and  intellectual 
framework  for  the  information.  Wurman  (19S9)  contends  that  the  ways  of  organizing  and 
structuring  information  are  finite.  He  has  identified  five  organizational  schemes:  category, 
time,  location.alphabet,  or  continuum.  A  major  advantage  for  the  use  of  multimedia 
resources  is  that  they  are  often  organized  in  a  way  that  mimics  the  associative  properties  of 
human  memory,  facilitating  the  user's  ability  to  create  personalized  organizational 
schemata  for  the  information  presented  (Jonassen  &  Wang,  1990).  However,  a  major 
disadvantage  is  that  without  system  control  and  learning  guidance,  there  is  a  greater 
danger  of  users  skipping  over  important  information  or  exiting  the  system  before  relevant 
content  is  accessed  (Milheim  &  Azbell,  1988). 

Attempts  to  measure  learning  during  research  activities  in  which  information  access 
in  not  tightly  controlled  defy  the  use  of  traditional  instruments  (e.g.  multiple  choice  tests). 
May  of  the  studies  that  compare  learning  with  multimedia  or  print  resources  compare 
amount  of  information  retained  rather  than  exploring  how  learners  cognitively  structure 
learned  information. 

Users  of  multimedia  resources  may  select,  store,  and  organize  information  in  their 
memories  in  a  variety  of  ways.  They  may  choose  to  attend  to  some  information  while 
ignoring  other  information.  This  means  that  methods  for  assessing  both  prior  knowledge 
and  knowledge  acquisition  when  learning  from  information  sources  must  be  used  that  "map" 
the  knowledge  of  users  to  better  understand  their  cognitive  schema.  The  current  study  used 
a  technique  similar  to  cognitive  mapping  known  as  "pattern  noting11  (Buzan,  1974)  to 
determine  users*  prior  and  subsequent  knowledge  when  using  multimedia  versus  print 
information  resources. 

Research  Questions 

1 .  What  information  location  and  use  activities  (i.e.  finding,  engaging,  extracting)  are 
utilized  when  accessing  information  using  a  print  or  multimedia  information  resource  and  do 
they  differ  due  to  the  age  of  the  learners? 

2.  Are  there  differences  in  motivation  (values  and  expectancies)  when  completing  a 
research  task  using  print  or  multimedia  resources? 

3.  In  what  ways  do  resulting  learning  patterns  differ  when  using  a  print  or  multimedia 
resource? 

These  questions  were  explored  for  children  using  cither  a  print,  or  multimedia  resource. 
Additional  data  were  gathered  from  adults  using  the  multimedia  resource  only. 


METHODS 

Instruments 

In  order  to  determine  information  location  and  use,  motivation,  and  learning, 
patterns  for  subjects  using  print  or  multimedia  resources,  three  types  of  instruments  were 
used. 

Observation  Checklist.  An  observation  checklist  was  developed  to  record  information 
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location  and  use  activities  exhibited  by  subjects  for  both  multimedia  and  print  treatments. 
These  activities  corresponded  to  computer  program  menu  choices  or  print  resource  options 
(e.g.  using  the  index,  using  the  table  of  contents,  random  browsing,  searching  by  topic  or 
keyword).  In  addition,  each  of  these  activities  was  classified  according  to  the  information 
location  and  use  skills  defined  by  Eisenberg  &  Berkowitz  (1990):  (1)  finding  information 
within  an  information  source  (browsing),  (2)  engaging  information  within  an  information 
source  (viewing,  listening,  and  reading),  and  (3)  extracting  information  from  an  information 
source  (notetaking). 

Motivation  Questionnaire.  The  Information  Resource  Attitude  Survey  (IRAS)  (Small, 
1992)  was  used  to  measure  subjects'  motivation  in  using  a  specific  resource.  Motivation  is 
further  defined  as  value  (how  interesting  and  relevant  is  the  information  and  resource)  and 
expectancy  for  success  (amount  of  confidence  in  the  resource  and  its  information  and  how 
satisfied  the  user  is  with  the  information  an/  1  -  source).  This  survey,  adapted  from  Keller's 
Instructional  Materials  Motivation  Survey  (i*  ^7),  was  used  to  assess  subjects'  motivation 
toward  using  a  specific  information  resource  (i.e.  a  book  or  an  interactive  videodisk  program) 
to  complete  an  assigned  task. 

The  IRAS  consisted  of  a  24-item  Likert-type  scale  with  two  subscales  of  twelve  items 
each  related  to  value  and  expectancy.  Subjects  were  required  to  rate  their  motivation  toward 
the  resource  from  1  (not  true)  to  5  (very  true).  All  items  were  identical  for  pre-  and  post- 
treatment  instruments  for  print  subjects  and  multimedia  subjects,  except  for  the  words 
"book"  and  "multimedia"  on  both  instruments  and  change  in  verb  tense  from  pretest  to 
posttest  (future  to  past  tense).  Some  examples  from  the  pretest  are: 

"I  expect  that  the  information  that  I  find  in  this  book  will  be  useful  to  me." 

(value) 

"I  believe  the  information  will  be  presented  in  an  exciting  and  stimulating 
way."  (value) 

"I  believe  that  this  book  will  be  easy  for  me  to  use."  (expectancy  for  success) 

"I  believe  that  I  will  have  control  over  the  type  and  amount  of  information  I  use  in 
this  resource."  (expectancy  for  success) 

A  Cronbach  coeffficient  alpha  was  used  to  test  the  reliability  of  the  motivation 
instrument.  The  overall  reliability  levels  for  the  pretest  was  .91  and  for  the  posttest  was 
.95.  The  reliability  levels  for  the  two  subscales  were  .73  (value  pretest)  and  .87  (value 
posttest)  and  .74  (expectancy  pretest)  and  .82  (expectancy  posttest)  which  were  deemed 
sufficiently  high  to  consider  both  subscales  reliable  (Nunally,  1978). 

Pattern  Noting,  When  completing  research  tasks,  learners  access  different  types  and 
amounts  of  information,  as  well  as  different  subject  matter.  Therefore,  there  is  little  control 
over  what  should  or  will  he  learned  and  traditional  tests  (e.g.  multiple  choice)  do  not 
adequately  assess  learning  in  these  situations.  One  method  that  has  been  advocated  for 
assessing  learning  under  these  circumstances  is  a  concept  mapping  approach  called  pattern 
noting. 

Pattern  noting  is  a  technique  used  to  assess  both  prior  knowledge  and  learning  by 
determining  changes  in  knowledge  structures  (Jonassen,  1987),  Pattern  notes  require  the 
individual  to  free-associate  with  a  topic  that  has  been  represented  by  a  word  or  phrase 
within  a  box  centered  on  a  blank  sheet  of  paper,  by  drawing  lines  containing  related  ideas 
(words,  phrases,  or  sentences).  Secondary  ideas  are  linked  to  primary  ideas  and  connections 
may  be  made  by  drawing  a  connecting  line  between  any  two  terms  that  are  perceived  as 
related.  Ideas  may  be  represented  as  words,  phrases,  or  complete  sentences.  Changes  from 
pro-treatment  to  post-treatment  pattern  notes  are  interpreted  as  indicators  of  learning 
(Small  &  (irabowski,  1992). 

The  number  of  main  terms  the  individual  writes  indicates  the  breadth  of  information 
that  individual  has  associated  with  the  given  topic  in  memory,  while  the  branch  terms  are 


an  indication  of  depth  of  knowledge.  Links  (connected  terms)  drawn  between  terms  indicate 
perceived  interrelatedness  of  terms  (Jonassen,  1987),  An  example  from  the  current  study 
appears  in  Figure  1 . 


Place  Figure  1  about  here 


Subjects 

Eighty-nine  middle  school  (grades  6,  7,  and  8)  students  served  as  initial  subjects  for 
this  study  and  were  randomly  assigned  to  one  of  the  two  treatments.  An  additional 
multimedia  treatment  group  consisting  of  36  adult  graduate  students  at  a  large 
northeastern  university  was  included  to  explore  potential  differences  in  results  due  to  age  of 
subjects.  All  subjects  completed  the  pretest  and  posttest  motivation  scales  and  pattern 
notes.  Each  subject  received  the  treatment  individually  while  being  observed  by  one  of  the 
researchers. 

Treatment 

A  topic  in  the  area  of  art  was  selected  as  the  content  area  for  the  materials  used  in 
this  study  because  this  was  thought  to  be  an  area  in  which  subjects  would  likely  have 
limited  knowledge  and  the. task  would  be  considered  relatively  novel  for  all  subjects.  Van 
Gogh  Revisited,  (Voyager  Company,  1988),  a  Hypercard-based  program  with  video  laserdisk, 
was  selected  for  the  multimedia  treatment.  This  program  contains  text  information  with 
slides,  photographs,  graphics  (timeline),  and  full  motion  video.  The  program  requires  two 
monitors,  one  for  text  information  contained  within  the  HyperCard  stack  and  the  other  for 
nontext  information  contained  on  the  videodisk,  a  format  that  facilitated  observation  of 
information  location  and  use  strategies.  The  program  also  offers  an  index  as  a  "find" 
mechanism,  a  "catalog"  which  simulates  a  table  of  contents,  and  arrows  that  allow  users  to 
progress  backwards  and  forwards  one  screen  at  a  time  which  approximates  browsing 
through  the  pages  of  a  book.  In  addition,  this  program  was  selected  because  researchers 
believed  that  subjects  would  have  little  or  no  prior  knowledge  of  the  content. 

The  hook  Vincent  Van  Gogh:  Art,  Life  and  Letters  (Zurcher,  1986,  Rizzoli  Publishers) 
was  chosen  as  the  print  treatment  because  it  was  closest  (except  for  full-motion  video  and 
audio)  in  formats,  content,  and  amount  and  type  of  information  to  the  multimedia  resource. 
It  contained  full-page,  full-color  art  prints  opposite  full-page  text,  a  timeline,  an  index,  and  a 
table  of  contents.  The  vocabulary  level  used  was  deemed  generally  equivalent  in  both 
resources. 

Procedures 

Prior  to  the  conduct  of  the  study,  all  subjects  were  trained  in  use  of  multimedia 
hardware  and  software  using  a  CD-ROM  -based  software  package  (Compton's  Multimedia 
Encyclopedia)  similar  in  formats  (e.g.  sound,  moving  images,  color,  graphics,  still  images, 
text)  and  functions  (e.g.  mouse-driven,  buttons,  icons)  to  the  one  used  in  the  treatment.  They 
were  also  trained  in  the  pattern  noting  technique.  Prior  to  the  treatment,  subjects  were 
administered  the  motivation  pre-treatment  questionnaire  and  were  asked  to  complete  an 
initial  pattern  note  on  the  life  and  art  of  Vincent  Van  Gogh  to  determine  their  prior 
knowledge  of  the  topic. 

Subjects  were  given  the  task  of  using  an  assigned  resource  (either  book  or 
multimedia)  to  gather  information  about  the  life  and  art  of  Vincent  Van  Gogh.  All  subjects 
wore  given  a  workspace  in  which  they  could  work  individually  and  undisturbed  for  30 
minutes.  An  observer  sat  with  each  subject  and  recorded  all  keystrokes  and  eye  movements 
on  the  observation  checklist.  Immediately  following  the  treatment,  subjects  were  asked  to 
complete  a  final  pattern  note  and  motivation  post-treatment  questionnaire. 
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RESULTS 

This  study  investigated  the  information  location  and  use  activities,  motivation,  and 
learning  patterns  of  subjects  using  either  a  print  or  multimedia  resource  to  complete  a 
research  task.  The  dependent  variables  in  this  study  were: 

*  number  and  type  of  information  location  and  use  activities 

•  responses  on  a  24-item  motivation  survey,  and 

the  number  and  type  of  post-treatment  pattern  noting  terms. 

Information  Location  and  Use 

Data  obtained  from  the  observation  checklists  were  analyzed  for  amount  and  type  of 
information  location  and  use  activities  (finding,  engaging,  extracting).  In  a  comparison  of 
children's  treatment  groups  using  Scheffe's  test,  those  using  the  multimedia  resource 
performed  significantly  more  finding  (F=8.1;  p<.0001)  and  engaging  (F=6.4;  p<.0001) 
activities  but  those  using  a  print  resource  performed  significantly  more  extracting  activities 
(F=22.6;  p<.0001)  (see  Table  1).  Analyses  that  determined  percentages  of  total  activities  in 
each  category  revealed  that  at  least  half  of  the  activities  in  both  treatment  groups  involved 
engaging  activities.  Print  subjects  spent  about  half  their  time  divided  between  finding  and 
extracting  activities  while  multimedia  subjects  spent  more  time  in  engaging  activities  and 
most  of  the  rest  of  their  time  in  finding  activities,  browsing  through  the  information. 


Place  Table  1  abcut  here 


A  comparison  of  the  two  multimedia  treatment  groups  found  that  adults  performed 
significantly  more  finding  and  engaging  tasks  than  children  but  not  more  extracting  tasks 
(see  Table  2).  However,  a  closer  look  at  the  data  by  percentage  of  total  activity  revealed  that 
the  results  were  quite  similar  for  both  groups. 


Place  Table  2  about  here 


A  further  analysis  of  engaging  activities  by  type  of  information  (text  or  nontext) 
indicated  that  children  in  the  multimedia  treatment  group  engaged  significantly  more 
nontext  information  than  children  in  the  print  treatment  group  but  that  both  groups 
engaged  a  similar  amount  of  text  (see  Table  3).  However,  an  analysis  by  percentage  of  total 
information  engaged  revealed  that  about  the  same  proportion  of  text  (55%)  and  nontext 
(45%)  was  engaged  by  print  subjects  but  three  times  as  much  nontext  (75%)  than  text  (25%) 
was  engaged  by  children  in  the  multimedia  treatment.  It  is  likely  that  the  availability  of 
video  in  the  multimedia  treatment  contributed  to  this  result. 


Place  Table  3  about  here 


A  comparison  of  both  multimedia  groups  revealed  that  although  adults  engaged 
significantly  more  of  both  text  (F=8.6;  p<.0001)  and  nontext  (F=5.8;  p<.001)  than  children, 
each  group  engaged  a  similar  ratio  of  nontext  to  text  (about  3-1)  by  percentage  (see  Table  4). 


Place  Table  4  about  here 


Motivation 

Because  the  novelty  effects  of  the  multimedia  treatment  might  have  influenced 
motivation  scores,  all  analyses  were  performed  on  the  basis  of  pretest-posttest  change 
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scores.  An  analysis  of  variance  on  IRAS  scores  indicated  that  there  were  no  significant 
differences  in  total  motivation  or  value  subscale  change  scores  between  children's  treatment 
groups.  There  was  a  significant  decrease  on  the  expectancy  subscale  for  the  print  treatment 
group  (F=5.2;  p<.02).  (See  Table  5)  Actual  mean  scores  decreased  from  pretest  to  posttest 
and  both  subscales  for  print  treatment  subjects  but  increased  on  total  test  and  value 
subscale  for  those  in  the  multimedia  treatment  group.  Multimedia  treatment  group  had 
significantly  higher  total  test  means  (F=7.4;  o<.05)  and  value  subscale  (F=8.4;  p<.05). 


Place  Table  5  about  here 


An  analysis  of  variance  for  adult  and  children's  multimedia  treatment  group  scores 
found  test  mean  scores  were  significantly  higher  for  adults  on  total  test  and  value  subscale 
(see  Table  6).  An  analysis  of  pretest  to  posttest  change  scores  found  adult  change  scores 
approached  significance  (decrease)  on  the  expectancy  subscale  (F=3.7;  p<.06).  Actual  scores 
showed  a  decrease  on  total  test  and  expectancy  subscale  for  adults  and  increased  on  total 
test  and  value  subscale.  Adult  mean  scores  on  the  total  test  score  means  (F=6.7;  p<.05)  and 
value  subscale  (F=19.9;  p<.01)  were  significantly  higher  than  children's  scores, 


Place  Table  6  about  here 


As  they  were  leaving  the  treatment  location,  some  adult  subjects  expressed 
frustration  and  uncertainty  about  the  amount  of  information  available  for  access  when  using 
a  multimedia  information  system.  Information  overload  may  negatively  affect  ability  to 
"integrate  and  cope  with  all  that  information. ..lead(ing)  to  dissatisfaction"  (Cummings, 
O'Connell  and  Huber,  1976,  p.  234),  This  phenomenon  may  have  contributed  to  a  lack  of 
subjective  certainty  about  the  adequacy  of  the  information  search  process  or  the  information 
accessed,  resulting  in  a  perceived  lack  of  closure. 

Learning 

For  the  children's  treatment  groups,  two  one-way  analyses  of  variance  of  the  posttest 
scores  were  conducted  separately  for  grade  level  and  gender  and  found  no  significant 
differences.  Multivariate  analysis  of  variance  revealed  no  significant  differences  between 
treatment  groups  in  terms  of  scores  on  the  Stanford  Achievement  Test,  8th  edition,  Form  J, 
a  standardized  achievement  test.  Therefore,  posttest  differences  between  treatment  groups 
could  be  considered  a  function  of  treatment  conditions.  Pre-treatment  and  post-treatment 
pattern  notes  were  analyzed  for  total  number  of  terms,  number  of  main  terms  (i.e.,  those 
connected  to  the  topic  term)  and  number  of  branch  terms  (i.e.,  those  connected  to  the  main 
terms). 

An  analysis  of  pattern  notes  for  both  children's  treatment  groups  indicated  no 
significant  difference  in  total  number  of  terms,  main  terms,  and  branch  terms  (see  Table  7). 
A  closer  look  at  the  proportion  of  main  to  branch  terms  by  percentage  showed  about  half  of 
the  terms  were  main  and  branch  terms  on  both  the  pretest  and  posttest  for  both  groups. 
However,  further  analysis  by  type  of  information  (words,  phrases,  sentences)  revealed  that 
on  the  posttest,  for  children  using  a  print  resource,  25%  of  the  terms  they  used  were  words 
while  most  (75%)  were  phrases  and  sentences  while  on  the  same  test,  40%  of  the  terms 
used  by  children  in  the  multimedia  group  were  words  and  a  majority  (60%)  were  phrases 
and  sentences.  However,  although  both  groups  had  more  words  than  phrases  or  sentences 
on  the  pretest,  the  print  group  used  more  phrases  and  sentences  than  words  on  the  posttest 
while  the  multimedia  group  continued  to  use  more  words  than  phrases  or  sentences, 
although  their  percentages  were  much  closer  for  all  three  types. 


Place  Table  7  about  here 


In  a  comparison  of  pattern  notes  for  subjects  in  both  multimedia  treatment  groups, 
adults  had  a  significantly  higher  number  of  terms  on  the  pretest.  A  comparison  of  pretest 
and  posttest  results  showed  within-group  increases  (i.e.,  the  total  number  of  terms,  main 
terms,  and  branch  terms  from  pretest  to  posttest)  for  all  treatment  groups.  As  expected, 
adults  used  a  significantly  greater  total  number  of  terms  (F=3.8;  p<.03)  and  number  of 
branch  terms  (F=10.2;  p<.O0Ol)  while  number  of  main  terms  approached  significance 
(F=2.5;  p<.Ofi)  on  the  posttest  (see  Table  8).  Since  adults  had  significantly  higher  scores  for 
both  pretest  and  posttest,  further  analyses  of  pretest  to  posttest  change  scores  was 
conducted.  They  indicated  a  significant  change  for  branch  terms  only  (F=22.77;  p<.0001) 
with  an  odult  mean  change  score  of  +10.1  and  children's  mean  change  score  of  +2.8. 


Place  Table  8  about  here 


Additional  analyses  of  the  data  were  performed  to  determine  the  percentage  of 
knowledge  levels  (i.e.,  main  and  branch  terms )  and  knowledge  representation  (i.e.,  words, 
phrases,  sentences)  on  both  the  pre-and  post-treatment  pattern  notes.  Results  indicated 
that  for  knowledge  representation,  both  groups  showed  an  increase  in  percentage  of  phrases 
and  sentences  and  a  decrease  in  percentage  of  words.  On  the  posttest,  40%  of  the  terms 
used  by  children  were  words,  while  60%  were  phrases  and  sentences,  while  for  adults,  63% 
of  the  terms  were  words  and  37%  were  phrases  and  sentences.  For  the  levels  subscales, 
adult  subjects  showed  a  larger  decrease  in  main  terms  (42%  to  26%)  than  children  (53%  to 
50%)  and  a  larger  increase  in  branch  terms  (58%  to  74%)  than  children  (46%  to  49%)  from 
pretest  to  posttest.  On  the  posttest,  both  children's  groups  had  about  the  same  percentage 
of  main  and  branch  terms  on  pretests  and  posttests  but  on  the  posttest,  adults  represented 
about  3/4  of  their  acquired  knowledge  as  branch  terms. 

DISCUSSION 

The  primary  research  questions  for  this  study  explored  the  information  location  and 
use,  motivation,  and  learning  patterns  for  young  and  adult  subjects.  Subjects  were  given  a 
research  task  using  either  a  print  or  multimedia  resource.  Researchers  recorded  information 
location  and  use  strategies  (finding,  engaging,  extracting)  on  an  observation  checklist.  The 
Information  Resources  Attitude  Survey  (IRAS)  evaluated  subjects'  motives  and  values 
related  to  the  resource  used.  A  pattern  noting  technique  was  utilized  to  assess  prior 
knowledge  and  learning  outcomes. 

More  finding  and  engaging  activities  were  performed  by  multimedia  treatment 
groups  than  print  treatment  group.  More  extracting  activities  were  performed  by  print 
subjects.  The  majority  of  activities  were  engaging  activities  for  all  groups.  Subjects  using  the 
print  resource  located  and  used  about  the  same  amount  of  text  and  nontext  information 
while  those  using  the  multimedia  resource  located  and  used  about  three  times  as  much 
nontext  as  text.  Although  adults  engaged  significantly  more  nontext  than  children,  the  ratio 
of  nontext  to  text  was  about  the  same  for  both  multimedia  groups. 

One  adult  subject  commented  after  the  treatment  that  with  a  book,  she  could  tell 
how  much  information  she  had  not  accessed  but  with  multimedia  because  she  could  move 
around  freely,  she  did  not  know  how  much  information  she  had  access  »d,  if  she  had 
accessed  all  the  information,  and  felt  dissatisfied  because  she  had  not  achieved  closure. 
Closure  has  been  considered  m  a  possible  contributing  factor  to  satisfaction  (Small  & 
Venkatesh,  1994). 

Results  indicated  that  both  children  and  adults  have  about  the  same  motivation 
toward  both  resource's.  Actual  differences  indicated  that  motivation  increased  for  children 
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following  the  multimedia  treatment  hut  not  for  adults  nor  for  children  using  a  print  resource. 

As  in  previous  studies,  book  users,  although  they  have  much  more  training  and 
experience  in  using  print  materials,  learned  no  more  than  those  using  a  computer-based 
multimedia  resource.  There  were  some  differences  in  learning  patterns  in  ways  knowledge 
was  represented  ;  i.e.,  multimedia  subjects  used  a  greater  percentage  of  single  word  terms 
while  print  subjects  used  a  greater  percentage  of  phrases  and  sentences.  This  may  be  the 
result  of  a  more  fragmented  engagement  of  information  when  using  a  multimedia  resource 
since  those  subjects  tended  to  do  more  browsing  ,  reading,  and  viewing.  Future  research 
may  reveal  that  users  of  multimedia  resources  spend  less  time  on  any  one  screen  and  more 
time  moving  around  in  the  information,  lending  to  more  of  n  "soand  bite"  learning 
experience.  These  results  indicate  the  need  for  better  methods  thnt  assess  knowledge 
acquisition  in  situations  where  much  of  the  information  learned  is  visual  and/or  auditory. 
Text-based,  verbal  methods  may  not  adequately  measure  learning  in  these  situations. 
Perhaps  more  qualitative  metuods  would  be  appropriate. 

Some  differences  between  adult  and  child  subjects  may  be  attributed  to  relevance  of 
the  task.  In  addition,  children  were  randomly  assigned,  while  jidultH  were  a  convenience 
sample. 

Another  interesting  outcome  was  that  subjects  in  the  multimedia  treatment  group 
repeated  more  inaccurate  information  (e.g.,  "Vincent  Van  (Jogh  invented  the  helicopter"  or 
"Vincent  Van  Gogh  painted  the  Momi  Lisa")  on  both  pattern  notes,  perhaps  indicating  that 
they  found  no  information  to  refute  these  ideas,  therefore  they  considered  them  accurate. 

Research  that  explores  both  affective  and  cognitive  outcomes  from  using  single 
format  and  multiple  format  (multimedia)  information  resources  may  help  designers  of 
multimedia  resources  and  teachers  provide  appropriate  guidance  for  their  use.  The  models 
for  conducting  independent  research  tasks  using  print  resourcews  may  not  be  appropriate  for 
the  use  of  multimedia  research  tools.  This  has  implications  not  only  for  multimedia 
designers,  but  for  teachers  and  school  library  media  specialists  who  are  charged  with  the 
responsibility  of  preparing  our  future  researchers. 

(Figures  and  tables  found  in  Appendix  G) 
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Abstract 

This  research  experiment  studied  the  effect  of  humor  versus  non- humor  on  learning 
and  retention  of  a  computer-based-instructional  lesson  on  tick  identification.  The 
experiment  also  surveyed  the  subjects'  enjoyment  of  the  lesson  material,  their  personal 
experiences  with  ticks  and  their  concerns  about  ticks  and  tick  borne  diseases. 
Undergraduate  students  from  a  statistics  class  (N=115)  were  randomly  assigned  to  one  of 
two  treatment  groups  or  the  control  group.  The  subjects  were  given  a  delayed  post  test  one 
week  after  the  treatment.  The  control  group  received  the  test  only.  The  results  indicated  no 
flignificant  difference  between  treatment  groups  in  terms  of  learning  and  retention  or  in 
terniH  of  enjoyment.  However,  the  humor  group  was  significantly  more  worried  about  ticks 
than  the  non-humor  group.  Implications  of  these  results  are  discussed. 
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INTUOIHJC  TION: 


"Tin1  vmuiik  Iwivi'  it  IfMt  i-HHV  nowadays.  In  my  days  you  were  thrashed  until  you  learnt 
.somrthmK   Why  nin'1  Ihry  li»arn  things  the  hard  way,  as  we  used  to?"  Fleming  (1966).  In 
response  to  such  }>r<m<mmcnu>ntH,  the  authors  of  this  paper  wish  to  take  issue.  We  think  it 
is  possible  to  provide  i*ffi*<*tiv<f  instruction  in  ways  that  encourages  students  and  makes  the 
learning  proc  ess  m  joy  a  hie,  especially  in  these  days  of  emerging  technologies.  The  strategy 
we  focused  on  to  test  this  opinion  was  the  effect  of  humor  on  learning. 

Before  the  mid- 1970*8  few  books  were  written  on  humor  as  a  topic  for  serious  research,  and 
the  majority  of  published  works  focused  on  humor  of  youngsters  and  adolescents.  In  1976 
the  first  International  Conference  on  Humor  and  Laughter  took  place  in  Cardiff,  Wales. 
This  was  met  with  considerable  amusement  and  ridicule  by  the  British  press  and  others 
who  pointed  out  the  absurdity  of  scholars  studying  humor.  They  felt  that  to  analyze  humor, 
like  analyzing  love,  was  to  risk  destroying  it  (McGhee,1989). 

However,  subsequent  international  conferences  followed  in  cities  around  the  world, 
including  the  first  annual  WHIM  Conference  (Western-later  called  World-  Humor  and  Irony 
Membership  Conference)  in  Arizona  in  1980.  Specialty  conferences  and  weekend  workshops 
also  emerged  bringing  public  awareness  to  the  value  of  humor  as  therapy  and  providing 
techniques  to  develop  "humor  skills"  (McGhee,1988). 

Powell  and  Anderson  (1985)  cite  more  than  70  studies  covering  various  aspects  of  humor 
and  teaching,  such  as  the  value  of  humor,  managing  undesirable  behavior,  building  self 
confidence,  enhancing  the  quality  of  students'  lives.  However,  in  their  review,  the  authors 
found  few  empirical  studies.  Those  that  do  exist  investigate  the  effect  of  humor  on  a  variety 
of  dependent  measures  with  mixed  results. 

In  general  the  literature  on  the  use  of  humor  in  education  can  be  categorized  into  three  main 
areas  of  investigation-achievement,  attention  and  interest,  and  positive  attitudes: 

Achievement: 

Studying  the  cognitive  "demands"  of  humor  on  children  Zigler,  Levine  and  Gould  (1967) 
suggest  that  there  is  much  gratification  in  the  cognitive  process  involved  in  responding  to 
humorous  stimuli.   There  is  a  sense  of  achievement  by  "seeing  the  joke."   This  suggests  an 
intrinsic  motivation  for  humor  in  learning. 

Hauck  and  Thomas  (1972)  found  with  elementary  school  children  there  was  an  increase  in 
recall  of  incidental  but  not  intentional  material;  however,  intentional  learning  was  found  by 
dabby  (1979)  to  be  significantly  higher  among  "low  creative"  students  using  humorous 
captions.  Studying  the  effect  information  acquisition  in  textbooks  using  humorous 
illustrations,  Bryant,  Brown,  Silverberg  and  Elliot  (1980)  found  no  differences. 

Kaplan  and  Pascoe  (1977)  found  overall  test  performance  in  university  students  was  not 
significantly  different  between  humorous  treatment  and  serious  treatment  groups,  but  there 
was  better  recall  of  humorous  examples.  Davies  and  Apter  (1980)  studied  285  primary 
grade  school  children  who  were  shown  a  series  of  tape-slides.  The  material  incorporating 
humor  resulted  in  the  greatest  recall  of  information.  However,  in  a  similar  experiment  with 
university  students  Clark  (1983)  found  non-significant  results  on  retention. 
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Ziv  (  1988a, ,1988b.)  found  significant  differences  in  favor  of  learning  with  relevant  humor 
compared  to  no  humor  in  two  studies  of  university  students  (the  second  replicated  the  first)  . 
In  each  case  his  research  was  based  on  one-semester  courses  as  opposed  to  studies  by 
others  whose  experiments  ranged  from  seven  minutes  to  one  hour. 

Attention  and  Interest; 

The  effects  of  humor  in  educational  television  as  an  influence  for  programming  selection  by 
first  and  second  graders  was  explored  by  Zillman  and  Bryant  (1983).  The  study 
demonstrated  that  "fast-paced  interspersion  of  humor  proved  more  effective  in  attracting 
and  sustaining  an  audience.. .than  the  interspersion  of  humor  at  a(n  intermediate  or)  slow 
pace"  (p.  175).  Schleicher  and  Bryant  (1982)  obtained  similar  results  in  their  study  of  high 
school  seniors, 

Gruner  (1970)  and  Markievicz  (1974)  found  the  effect  of  gaining  audience  attention  and 
interest  in  a  particular  topic  is  enhanced  with  humor,  although  humor  did  not  appear  to 
affect  comprehension  and  acceptance  of  a  message.  Warnock  (1989)  emphasized  that  humor 
is  a  good  means  of  keeping  students  attentive.  "Unchecked  boredom  can  make  the  best  of  us 
mediocre.  A  timely  joke  or  recounting  an  amusing  happening  can  go  a  long  way  toward 
freshening  up  a  dull  meeting  and  putting  students'  minds  back  on  track,.. "(p.23.) 

Positive  Attitude: 

Fleming  (1966)  rationalized  that  humor  in  pictures  and  text  can  aid  in  developing  positive 
attitudes  and  learning  in  students  of  modern  languages,  much  like  political  cartoons  can 
"sway  an  entire  election  campaign."  Testing  this  theory,  Bryant,  Brown,  Silverberg  and  Klliot 
(1980)  studied  the  effects  of  humorous  illustrations  in  textbook  material  on  undergraduate 
students.  The  findings  based  on  various  dependent  measures  were  mixed.  However,  the 
material  with  humorous  illustrations  was  considered  to  be  more  enjoyable. 

Sewell  (1979)  determined  that  "if  comprehension  is  the  only  goal  of  instruction,  the  less 
expensive  printed  text  is  just  as  effective  as  the  more  expensive  illustrated  text  or  audio- 
visual presentation."  (p.508.)  However,  his  study  indicated  increased  student  enjoyment  of 
material  that  incorporated  humor,  and  he  concluded  that  the  decision  of  which  presentation 
format  to  use  should  be  based  on  personal  preference  and  budget  constraints. 

Ziv  (1979)  found  that  humor  plays  a  significant  role  in  creating  a  positive  class  atmosphere. 
One  of  the  qualities  students  would  most  hope  to  find  in  their  teachers  is  humor.  According 
to  Hill  (1988),  "One  of  the  most  important  functions  of  humor  (in  the  classroom)  is  to  create 
a  positive  learning  environment. "...When  students  can  relate  what  they  learn  to  a 
memorable  context,  whether  it  is  visual  or  emotional,  they  are  more  likely  to  remember  the 
information.  Using  jokes  and  anecdotes  to  enhance  stories  provides  such  an  association." 
(p.20). 


PURPOSE  OF  THE  INVESTIGATION: 

Much  research  has  been  conducted  on  various  aspects  of  the  traditional  teaching  and 
learning  methods;  however,  with  the  emergence  of  sophisticated  technologies  new  questions 
arise.  Computers  are  increasingly  integrated  into  total  learning  programs-  as  a  means  of 


accessing  data  bases,  providing  additional  practice  and  guidance  for  various  learning 
objectives,  and  as  stand-alone  lessons  for  independent  learning  and  distance  education. 

Unfortunately,  up  till  now,  much  computer  courseware  design  has  been  less  than  innovative. 
The  results  and  presentations  are  simply  electronic  workbooks,  emphasizing  rote  learning 
rather  an  aid  to  higher  level  thinking.  In  addition,  some  people  are  reluctant  to  use 
computers  for  any  purpose  because  they  feel  the  technology  is  alien  and  unfriendly. 

One  strategy  used  by  some  educators  to  create  a  pleasant,  inviting  atmosphere  in 
traditional  learning  environments  is  humor.  However,  humor  as  a  topic  for  serious  research 
has  been  minimal,  and  there  have  been  no  studies  on  humor  as  a  strategy  in  Computer  - 
Based-Instruction  to  enhance  learning  and  retention. 

We  hypothesized  that  a  strategy  incorporating  a  humorous  theme  and  humorous  comments 
relevant  to  the  content  would  help  create  a  positive  learning  atmosphere,  motivate  students 
to  attend  to  the  material  and  aid  in  encoding  facts  and  concepts  better  than  material 
presented  in  a  standard,  non-humorous  way. 


DEFINITION: 

What  is  humor?  And  what  is  it  to  be  humorous?  McGhee  (1979)  traces  the  Latin  term 
"humor"  to  ancient,  medieval  and  Renaissance  physiology  when  it  referred  to  one  of  four 
bodily  fluids  associated  with  temperament.  To  be  in  a  "good  mood"  meant  the  fluids  were 
balanced;  otherwise,  one  would  be  "out  of  humor."  (p.5).  Later  the  term  "humorist"  was 
applied  "to  anyone  who  was  highly  skilled  at  producing  amusing,  incongruous,  ridiculous  or 
ludicrous  ideas  and  events."  (p.5).  Freud  believed  humor  to  be  important  as  a  coping 
mechanism  (MacHovec  1988).  Freud  further  maintained  that  humor  permitted  adults  to 
enjoy  a  childlike  release  from  societal  restrictions  on  behavior  (Keith-Spiegel,  1972).  Sorrell 
(1972)  states  "laughter  lifts  man  above  his  animalistic  state,  sets  him  free,  and  gives  his 
spirituality  another  dimension."  (p.2).  According  to  MacHovec  (1988)  humor  is  a  universal 
characteristic. 

From  ancient  to  modern  times,  regardless  of  culture,  religion,  geographic  location, 
language,  ethnic  identity  or  gender,  laughter  is  a  part  of  everyone's  experience.  MacHovec 
(1988)  calls  it  "a  complex  psychological-emotional  phenomenon."  (p.3).  Although  what  is 
considered  funny  for  one  person  or  group  may  not  be  so  for  another,  still  there  are  classic 
stories  and  situations  that  transcend  cultures  and  time. 


Definition  of  Humor  used  in  this  study:: 

In  reviewing  the  literature,  a  broad  range  of  definitions  f  humor  was  uncovered.  However,  to 
best  operationalize  the  term,  so  as  to  distinguish  the  dnrerences  between  treatments,  the 
following  specific  attributes  were  selected: 

A  light-hearted  presentation  of  material,  rather  than  a  facts-only  scientific  presentation; 
inclusion  of  whimsical,  content  related  cartoons  and  animation;  the  use  of  a  theme  that  is 
ridiculous,  exaggerated  and  narrated  by  a  character  using  an  informal,  conversational  style. 


MKTHOI): 


Subjects  and  Setting 

One  hundred  and  fifteen  volunteer  undergraduate  students  of  various  majors  enrolled  in  a 
statist  ic*  coiirw  .it  a  majtir  state  university  participated  in  this  study.  Each  subject  was 
randomly  ahMghed  t c »  *jm«*  of  thrc»«*  groups. 

A.  Computer  Bused- Instructional  (CB1)  Lesson  on 
tick  identification  presented  with  humor 

II  C "lil  Lesson  on  tick  identification  presented  without  humor 

C     Cunt  ml 

Varimih  lab  tunes  were*  scheduled  for  student  conveniences.  The  subjects  were  randomly 
assigned  to  one  of  two  treatment  groups  or  control  when  they  arrived  at  the  lab.  Attendance 
varied  at  the  lab  sessions  from  as  many  as  twenty  people  to  fewer  than  two, 

Trout  mo  i\t 

A  computer  based  instructional  lesson  (CBI)  entitled  "The  Anatomy  of  a  Hard  Tick"  was 
used  as  the  treatment  basis  for  this  investigation.  The  rationale  for  selecting  this  topic  was 
that  it  provided  a  rich  source  of  facts  and  concepts;  it  also  was  a  relevant  topic  to  a  health 
problem  of  growing  importance- -Lyme  and  other  tick-borne  diseases. 

The  CBI  lesson  was  presented  to  two  of  the  treatment  groups.  The  first  group,  comprising 
-Hi  students,  received  the  lesson  with  the  inclusion  of  a  humor  as  defined  above.  The  second 
group,  comprising  l\2  students,  received  the  CBI  lesson  without  the  inclusion  of  any  humor. 
A  third  group,  comprising  40  students,  served  as  a  control  and  received  no  lesson. 

Validation  of  Material  and  Test 

To  assess  the  degree  of  correspondence  between  the  delayed  post-test  questions  and  the 
treatment  content,  four  college  science  educators  compared  each  of  the  fourteen  test 
questions  in  terms  of  their  appropriateness  and  clarity  and  whether  or  not  the  questions 
should  be  changed.  The  types  of  questions  included  matching,  multiple  choice  and  short 
answers.  The  panelists  were  in  basic  agreement  on  all  questions.  Disagreement  occurred 
only  as  to  whether  to  include  more  terms  than  numbers  on  questions  requiring  matching 
terms  to  numbers. 

Validation  of  the  Treatments 

A  panel  of  four  impartial  individuals  who  weie  either  faculty  members  of  the  university  or 
graduate  students,  assessed  55  screens  of  the  material  to  be  used  in  the  humor  treatment 
group  for  the  degree  of  humor  they  contained  based  on  the  definition  of  humor  described 
earlier.  A  Lickert-type  scale  of  one  to  three  was  used  with  three  being  the  highest  amount  of 
humor.  The  mean  average  of  these  screens  was  2.4.  Screens  having  identical  material  for 
both  the  humor  and  non-humor  groups  were  not  evaluated. 

Dcpen  den  t  Meas  u  res 

ACHIEVEMENT:  One  week  after  the  initial  treatments  were  administered,  all  three  groups 
were  given  identical  print-based  tests  on  factual  and  conceptual  knowledge  of  tick  anatomy. 
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There  were  36  responses  required  comprising  short  unsworn,  multiple  choice  and  matching 
numbers  to  parts. 

ATTITUDE:  In  addition  the  subjects  wore  also  tfivni  H\\  h  item  background  and  opinion 
survey  to  assess  the  influences  of  their  background  and  cxpcricncon  with  ticks  on  their 
knowledge  acquisition  of  the  treatment  materia!  and  their  response  to  the  lesson. 
Questions  included  college  major,  place  of  residence     items),  personal  "relationship"  with 
ticks  and'or  Lyme  disease  (2-items),  fear  of  inserts,  worry  about,  ticks  and  tick-home 
diseases  and  enjoyment  of  the  lesson. 


RESULTS: 

Delayed  Post-Test  Performance 

Statistical  one  factor  analysis  of  variance  was  lined  to  determine  the  difference  between 
groups  in  terms  of  total  scores  on  the  delayed  post -lest   No  significant  difference;  was  found 
between  the  humor  and  non-hiunor  groups,  only  between  the  control  group  and  treatment 
groups.  Both  treatment  groups  were  equal  in  terms  of  enjoyment  of  the  lessons. 
( See  Figure  1.) 

A  two-factor  analysis  of  variance  was  used  to  determine  the  differences  in  total  scores  based 
on  having  been  bitten  by  a  tick  or  having  con :r acted  Lyme  disease.  The  prediction  for  this 
comparison  was  that  personal  experience  with  ticks  may  increase  interest  and  attention  in 
the  lesson  material.  However,  no  differences  were  found.  In  addition,  whereas  certain 
regions  of  the  country  are  more  vulnerable  to  ticks  and  tick-borne  diseases  than  others,  the 
location  of  the  participants'  home  residence  also  did  not  factor  into  the  results. 

On  the  affective  measure,  the  extent  to  which  participants  were  afraid  of  insects,  spiders 
and  ticks,  no  significant  difference  was  indicated  between  any  of  the  three  groups;  however, 
in  terms  of  the  degree  to  which  subjects  were  worried  about  ticks  and  tick-borne  diseases, 
there  was  a  significant  difference  at  p.<.05  with  the  humor  group  being  more  worried.  (See 
Figure  2.) 


TABLE  1.  Comparison  of  Treatment  Groups  for  Achievement 

Treatment       N         Mean  s_d 

No  Humor  32  28.37  6.83 
Humor  4328.60  6.22 

Control  40        13.90  6.40 


TABLE  2.  Comparison  of  Treatment  Groups  for  Worry  about  Ticks 
and  Tick-borne  Diseases 


Treat 

N 

Mean 

sd 

No  Humor 

32 

2.63 

1.39 

Humor 

43 

3.51* 

1.28 

Control 

40 

2.85 

1.37 

*  p.<  05 
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DISCUSSION: 


The  results  of  this  study  indicate  that  there  were  no  significant  difference  found  between 
treatment  strategies  in  terms  of  learning  and  retention.  These  findings  are  supported  by 
Sewell  (1979).  His  study  compared  comprehension  and  evaluation  of  five  different  treatment 
strategies  using  cartoons  that  accompanied  textual  material. 

The  authors  speculate,  however,  that  the  scientific,  sequential  use  of  graphics  was 
sufficiently  meaningful  to  produce  highly  positive  results  equivalent  to  the  humor  group. 
These  results  are  encouraging  in  that  allowing  the  learner  to  choose  the  type  of  instruction 
would  not  compromise  the  quality  of  learning. 

In  terms  of  the  affective  results  of  the  study,  the  humor  group  indicated  significantly  more 
worry  about  ticks  and  tick-borne  disease.  This  finding  is  important  since  being  concerned 
about  potential  dangers  may  imply  that  precautions  are  more  likely  to  be  taken  in  areas 
prone  to  high  numbers  of  ticks  and  medical  advice  sought,  if  bitten.  These  data  also  may 
imply  that  another  form  of  learning  took  place  as  a  result  of  more  humorous  reinforcement  of 
concepts.  There  are  further  have  implications  in  terms  of  developing  strategies  for 
consciousness  raising  on  various  important  topics, 

IMPLICATIONS  for  FUTURE  RESEARCH: 

Questions  are  raised  as  a  result  of  the  current  study.  Would  separating  the  groups  into 
humor  and  non-humor  meeting  times  make  a  difference  to  learning  and  retention?  During 
this  study  only  one  third  of  the  subjects  received  humorous  material  at  any  session. 
However,  when  there  were  fewer  subjects  in  the  room,  or  only  one  subject  in  the  room,  there 
was  more  outer  expressions  of  amusement  observed,  such  as  smiles  and  laughs.  In  a  group 
situation,  perhaps  there  is  a  need  for  a  "critical  mass"  attending  the  same  types  of 
humorous  information  to  be  amused. 

Would  a  "warm-up"  presentation  help  set  the  tone?  This  is  a  standard  technique  in  the 
entertainment  industry  to  put  audiences  in  a  receptive  mood.  Should  humor  be  combined 
with  other  enhancements,  such  as  sounds?  Would  learners,  who  have  specific  and  direct 
need  for  the  knowledge,  show  significantly  different  results  in  achievement  and  attitude?  For 
instance,  in  a  lesson  on  tick  identification,  would  health  care  workers,  campers  and/or  field 
workers  score  higher  on  the  test  than  casual  learners? 

Would  treatments  that  were  more  diverse  be  a  better  means  of  assessing  of  the  effect  of 
humor?  In  attempting  to  keep  both  treatment  groups  as  equal  as  possible,  memory  aids 
were  presented  to  both  groups- -with  text  only  for  the  non-humorous  mnemonic  and  text  with 
cartoons  in  the  humorous  mnemonic.  Normally,  a  traditional  science  lesson  wuld  not  have 
memory  aids  provided. 

Would  it  make  a  difference!  in  achievement  and/or  attitudes,  if  learners  were  given  a  choice  of 
presentation  styles;  thus  allowing  for  learner  control.  Finally,  is  there  a  significant 
difference  in  knowledge  acquisition  and  retention,  between  scientific  drawings  presented  in  a 
complete  form  rather  than  in  a  developmental  sequence?  The  use  of  developmental 
progression  of  graphic  displays  for  both  treatment  groups  in  this  study  may  have  influenced 
the  results.  Follow-up  investigations  are  planned  to  address  some  of  these  questions. 


CONCLUSION: 

The  theory  behind  designing  computer-based- instructional  courseware  with  the  inclusion  of 
humor  was  an  attempt  to  recreate  the  feeling  one  might  experience  in  a  warm  and  inviting 
classroom.  Creating  a  friendly  atmosphere  is  important  for  general  appeal,  and  for  learners 
wary  or  emerging  technologies  and  computers  in  particular,  increasing  the  comfort  level  of 
CBI  could  contribute  to  learner  acceptance  of  the  medium.  Learners  are  more  apt  to  use  and 
enjoy  CBI  programs  that  meet  individual  preferences,  and  incorporating  humor  can  affect 
attitudes  and  may  enhance  some  types  of  knowledge  acquisition. 
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The  purpose  of  this  paper  is  to  report  the  results  of  a  study  that  examined  the  role  of 
perceptions  in  the  instructional  development  and  adoption  process.  According  to  the  theories 
developed  by  Rogers  (1983),  if  potential  adopters  have  favorable  perceptions  of  an 
innovation's  attributes,  then  they  are  more  likely  to  adopt  the  innovation.  Many 
researchers  (e.g.,  Clinton,  1972;  Kivlin,  1960;  Weinstein,  1986;  Wyner,  1974)  have 
examined  the  role  that  perceptions  play  in  the  adoption  of  an  innovation.  The  present  study 
was  unique  because  it  examined  not  only  the  role  of  perceptions  in  the  adoption  of  an 
innovation,  but  also  the  role  that  perceptions  played  in  the  development  of  the  innovation. 

The  attributes  of  an  innovation  are  the  basis  upon  which  potential  adopters  form  their 
perceptions  of  the  innovation  (Rogers,  1983).  According  to  Rogers,  all  innovations  can  be 
thought  of  as  having  five  general  attributes.  These  attributes  are  relative  advantage, 
complexity,  compatibility,  observability,  and  trialability.  This  study  examined  the 
perceptions  of  potential  adopters  and  instructional  developers  in  regard  to  the  attributes  of 
three  computer-based  learning  modules.  In  addition,  the  role  that  perceptions  played  in  the 
development  and  adoption  of  the  modules  was  examined. 

Statement  of  the  Problem 

Many  of  the  products  and  practices  of  instructional  technology  have  failed  to  make  a 
major  impact  on  education.  Burkman  (1987)  writes  that  many  endeavors  that  use  the 
research-development-diffusion  (RDD)  paradigm  suffer  from  a  lack  of  utilization. 
Instructional  technology  is  an  endeavor  that  makes  extensive  use  of  the  RDD  paradigm. 
Burkman  (1987)  writes  that  instructional  technology  has  experienced  a  lack  of  utilization  in 
all  fields,  including  primary  and  secondary  schools,  colleges  and  universities,  and  even  in 
industry  and  the  military.  One  reason  for  this  lack  of  utilization  could  be  that  instructional 
developers  do  not  understand  and  account  for  the  perceptions  of  potential  adopters  during 
the  development  process.  Referring  to  educational  technologists  in  general,  Dal  ton  (1989) 
writes  that  "although  we  can  fill  instructional  gaps  with  fervor,  we  never  seem  to  examine 
our  solutions  in  light  of  the  wants  of  the  imple mentors"  (p.  22).  Instructional  developers 
commonly  believe  that  products  which  result  in  more  "effective  and  efficient"  instruction  will 
automatically  be  attractive  to  potential  adopters  (Burkman,  1987).  Ralph  W.  Tyler  (1980) 
writes  "many  developers  of  technology  accept  the  view  that  as  time  passes,  there  will  be 
increasing  use  of  the  innovation  until  it  has  become  a  common  element  in  school  practice"  (p. 
11).  From  these  statements,  we  can  see  that  many  developers  have  the  assumption  that  if 
they  develop  more  effective  instruction,  adoption  and  diffusion  will  follow  in  time.  This 
might  be  a  false  assumption. 
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Figure  1.  A  model  showing  how  knowledge  of  perceptions  can  affect  development  of  a 
product. 

Burkman  (1987)  feels  a  more  user-oriented  approach  to  instructional  development  is 
needed  to  increase  utilization.  He  also  believes  that,  as  part  of  this  user-oriented  approach, 
instructional  developers  must  seek  to  understand  and  positively  manipulate  the  perceptions 
of  potential  adopters  in  regard  to  the  attributes  outlined  by  Rogers  (1983).  A  more  user- 
oriented  approach,  one  that  seeks  to  understand  and  take  into  account  the  wants  of  the 
implementors,  might  lead  to  the  increased  use  of  instructional  innovations. 

Figure  1  is  a  model  showing  how  knowledge  of  the  perceptions  of  potential  adopters 
might  result  in  modifications  to  an  instructional  product.  These  modifications,  in  turn, 
might  lead  to  changes  in  the  perceptions  of  potential  adopters  in  regard  to  the  product.  This 
process  of  development,  knowledge  gaining,  and  modification  would  result  in  the  circular 
model  depicted  in  Figure  1.  The  use  of  such  a  model  by  instructional  developers  could  lead 
to  the  increased  adoption  and  use  of  their  instructional  products 

The  Adoption  Situation  in  This  Study 

The  study  described  here  was  part  of  a  larger  evaluation  study  of  three  multi  media 
computer-based  learning  (CBL)  modules.  The  three  multi-media.  CBL  modules  were 
developed  for  use  in  three  weather  forecasting  agencies.  The  modules  represent  the  first  in  a 
series  of  twenty-one  modules  to  be  developed  for  the  agencies.  The  modules  are  to  be  used 
as  on-site  training  for  operational  forecasters  at  multiple  sites  in  each  of  the  three  agencies. 
The  computer  delivery  method  and  the  independent  study  provide  enhanced  opportunities 
for  realistic  study  of  forecasting  knowledge  and  skills.  The  modules  are  an  innovative 
method  of  training  offered  on-site  to  weather  forecasters.  The  purpose  of  the  overall 
evaluation  was  to  determine  how  the  content,  strategies,  procedures,  and  implementation  of 
the  modules  might  be  modified  to  enhance  cognitive  and  affective  outcomes.  Information 
gathered  in  the  evaluation  of  the  first  three  modules  will  be  used  to  inform  the  design  of 
succeeding  modules. 

The  adoption  situation  examined  in  this  study  is  somewhat  unique  because  there 
were  two  groups  of  potential  adopters.  The  operational  forecasters  in  the  field  who  will  use 
the  modules  were  one  group  of  potential  adopters.  The  organizations  that  commissioned  the 
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production  of  the  modules  were  also  a  group  of  potential  adopters.  In  order  for  the  CBL 
modules  to  be  fully  utilized,  it  was  necessary  for  both  groups  to  adopt  the  modules.  The 
forecasting  organizations,  for  their  part,  had  to  commission  production  of  the  modules, 
accept  delivery  of  the  finished  products,  make  the  completed  modules  available  at  the  field 
sites,  and  encourage  their  forecasters  to  use  the  modules.  The  individual  forecasters,  for 
their  part,  had  to  routinely  use  the  modules,  devote  time  and  effort  to  studying  the  modules, 
and  incorporate  the  CBL  modules  into  the  daily  work  environment  of  the  forecasting  sites. 
Either  the  organizations  or  the  forecasters  could  have  prevented  the  adoption  and  full 
utilization  of  the  modules  by  failing  to  perform  the  activities  listed  above. 

For  purposes  of  this  study,  the  forecasters  were  the  potential  adopter  group  whose 
perceptions  were  examined  by  the  researcher.  The  forecasters  were  selected  for  study 
because  they  were  the  intended  end  users  of  the  modules.  Examining  the  perceptions  of  the 
end  users  linked  the  present  study  more  directly  to  Burkman's  (1987)  theory  that  a  more 
user-oriented  approach  to  instructional  development  could  increase  adoption  of  the 
innovation. 

Research  Questions 

This  study  sought  to  answer  four  research  questions: 

1)  What  were  the  perceptions  of  instructional  developers  in  regard  to  relative  advantage, 
complexity,  compatibility,  trialability,  and  observability  during  the  development  of  the 

computer-based  learning  modules? 

2)  What  were  the  perceptions  of  potential  adopters  in  regard  to  the  relative  advantage, 
complexity,  compatibility,  trialability,  and  observability  of  the  computer-based  learning 
modules? 

3)  If  instructional  developers  were  aware  of  the  perceptions  of  potential  adopters,  how 
did  that  awareness  affect  the  development  of  the  modules? 

4)  How  did  the  perceptions  of  instructional  developers  and  potential  adopters  compare  in 
regard  to  the  relative  advantage,  complexity,  compatibility,  trialability,  and  observability  of 
the  computer-based     learning  modules? 

Limitations  of  the  Study 

There  are  two  key  limitations  to  this  study.  The  first  limitation  is  that  this  study  did 
not  attempt  to  determine  the  importance  of  perceptions  relative  to  other  factors  which 
influence  adoption.  Rogers  and  Shoemaker  (1971)  write  that  five  variables  influence  an 
innovation's  rate  of  adoption  These  variables  are  perceived  attributes,  type  of  innovation- 
decision,  communication  channels,  nature  of  the  social  system,  and  the  extent  of  the  change 
agents'  promotion  efforts.  The  present  study  examined  only  the  perceived  attributes  of  the 
innovation. 

The  second  key  limitation  of  this  study  is  that  this  study  did  not  attempt  to  determine 
how  perceptions  influenced  the  long-term  retention  of  the  modules  after  their  adoption.  As 
Burkman  (1987)  points  out,  while  the  decision  to  adopt  an  innovation  is  based,  in  part,  on 
subjective  perceptions,  "once  an  innovation  has  been  adopted  .  .  .  perceplion  tends  to  be 
displaced  by  reality."  This  study  was  concerned  with  how  perceptions  affected  the  initial 
decision  (i.e.,  adoption  or  non-adoption). 


Review  of  the  Literature 
This  study  is  based  upon  two  key  literature  sources.  The  first,  and  primary  source,  is 
E.M.  Rogers'  writings  related  to  the  communication  and  diffusion  of  innovations.  Chief 
among  these  writings  are  Rogers  (1983)  Diffusion  of  Innovations  (3rd  edition)  and  Rogers 
and  Shoemaker  ( 1971)  Communication  of  Innovations:  A  Cross- Cultural  Approach  (2nd 
edition).  These  books  provide  the  theoretical  basis  for  many  of  the  studies  cited  in  this 
review,  especially  those  studies  primarily  concerned  with  the  perceived  attributes  of  an 
innovation. 

The  second  literature  source  upon  which  this  study  is  based  is  Ernest  Burkman's 
(1987)  chapter  entitled  "Factors  Affecting  Utilization"  that  appears  in  Instructional 
Technology:  Foundations  (Gagne,  1987).  In  this  chapter,  Burkman  calls  for  a  more  user- 
oriented  approach  to  instructional  design.  Part  of  this  new  approach  calls  for  instructional 
designers  to  determine  the  attributes  most  important  to  potential  adopters  and,  based  upon 
that  knowledge,  to  design  products  that  present  positive  perceptions  of  those  attributes 
(Burkman,  1987). 

Research  Related  to  Perceived  Attributes 

Hurt  and  Hibbard  (1989)  write  that  "it  is  well-documented  in  the  diffusion  research 
that  the  characteristics  of  innovations  as  perceived  by  potential  adopters  play  a  critical  role 
on  the  rate  of  acceleration  of  the  adoption  curve"  (p.  214).  Many  studies  of  perceived 
attributes  have  been  based  directly  on  Rogers1  theory  of  perceived  attributes.  A  variety  of 
fields,  ranging  from  agriculture  to  sociology,  are  represented  by  these  studies. 

The  first  large-scale  studies  to  examine  the  relationship  between  perceptions  and  an 
innovation's  rate  of  adoption  took  place  in  the  early  1960s.  The  chief  investigators  of  the 
relationship  at  that  time  were  Joseph  Kivlin  and  Frederick  Fliegel.  Kivlin's  ( 1960)  study  of 
the  perceptions  of  farmers  in  Pennsylvania  in  regard  to  numerous  agricultural  innovations  is 
the  earliest  exploration  of  perceived  attributes  found  in  the  literature.  Although  this  study 
predated  Rogers'  theory,  Kivlin's  attributes  are  largely  analogous  to  the  attributes  developed 
Rogers. 

Kivlin  found  that  relative  advantage,  complexity,  and  mechanical  attraction  were  the 
attributes  most  significant*;'  related  to  adoption  for  the  Pennsylvania  farmers.  Also  in  this 
study,  Kivlin  determined  tliat  the  characteristics  of  an  innovation  accounted  for  51%  of  the 
variability  in  rate  of  adoption.  Kivlin's  (1960)  study  presents  the  strongest  and  earliest- 
found  evidence  that  the  assumption  which  underlies  the  present  study  is  valid. 

In  1966,  Fliegel  and  Kivlin  studied  the  perceptions  of  the  same  229  farmers  as 
Kivlin's  1960  study.  In  the  1966  study,  Fliegel  and  Kivlin  found  that  trialability  was  the 
most  significant  attribute  related  to  adoption.  Relative  advantage,  one  of  the  most 
sife  ;ficant  attributes  in  the  earlier  study,  emerged  as  the  second  most  significant  attribute 
in  the  1966  study. 

Kivlin  and  Fliegel  (1967a)  also  studied  the  different  perceptions  that  small-scale 
farmers  and  middle-scale  farmers  had  in  regard  to  various  agricultural  innovations.  They 
found  that  small-scale  farmers  were  slower  to  adopt  new  practices  than  were  the  middle- 
scale  farmers.  The  researchers  concluded  that  the  difference  in  the  rate  of  adoption  was  not 
only  a  "function  of  production  scale  but  also  ...  a  result  of  differences  in  perceptions"  (p.  90). 
Kivlin  and  Fliegel's  (1967a)  study  is  important  because  it  is  the  first  example  of  a  study 
concluding  that  differences  in  the  perceptions  of  two  groups  account,  in  part,  for  differences  in 
rate  of  adoption.  Another  study  by  Kivlin  and  Fliegel  during  this  period  (1967b)  also  found 
that  relative  advantage  was  the  most  influential  attribute  related  to  rate  of  adoption. 
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Gurmeet  Singh  Sekhon  (1968)  studied  perceptions  of  farm  innovations  in  Punjab, 
India.  The  population  for  Sekhon's  study  was  made  up  of  professional  advocates  of  an 
innovation  and  their  clients.  Clients  were  divided  into  those  who  had  a  high  degree  of 
contact  with  the  professional  advocates,  and  those  who  had  low  contact.  Sekhon 
hypothesized  that  professional  advocates  and  their  high  contact  clients  would  share  similar 
perceptions  of  innovations.  Sekhon  found,  however,  that  the  two  groups  had  highly 
dissimilar  perceptions.  The  results  of  Sekhon's  study  are  similar  to  the  results  of  Ross* 
(1983)  study.  Ross  discovered  that  change  agents  and  non  adopters  in  his  study  shared 
perceptions  which  were  more  similar  than  the  perceptions  shared  by  change  agents  and 
adopters.  It  is  possible  to  hypothesize,  in  view  of  the  studies  by  Sekhon  and  Ross,  change 
agents  and  potential  adopters  do  not  have  to  share  similar  perceptions  in  order  for  an 
innovation  to  be  adopted. 

Clinton  (1972)  studied  the  role  that  perceived  attributes  played  in  the  acceptance  of 
innovations  by  337  teachers  in  Canada.  He  found  that  teachers  categorized  innovations 
according  to  the  attributes  they  perceived  and  that  the  actual  acceptance  of  the  innovation 
depended  upon  the  perceived  attributes.  Clinton  arrived  at  two  major  conclusions.  The  first 
was  that  how  teachers  perceived  an  innovation  was  as  important  as  the  innovation  itself. 
The  second  was  that  innovation  is,  during  its  early  stages,  a  mental  process  not  a  physical 
act. 

In  1974,  two  studies  (Wyner;  Hahn)  that  examined  the  theory  of  perceived  attributes 
were  published.  Wyner  (1974)  used  attributes  to  study  teachers'  perceptions  in  regard  to  an 
educational  innovation  in  an  elementary  school.  She  concluded  that  the  perceptions  of  those 
who  use  an  innovation  can  provide  valuable  data  to  change  agents.  Hahn  (1974)  studied 
the  perceptions  of  over  200  high  school  teachers  in  regard  to  educational  innovations. 
Observability  was  the  attribute  that  was  most  significantly  related  to  the  adoption  of  an 
innovation  in  Hahn's  study.  Hahn's  conclusion  is  fairly  unique  because  no  other  study 
located  in  this  review  of  the  diffusion  literature  found  observability  to  be  the  primary 
attribute  related  to  adoption. 

Keeler  (1976)  and  Holloway  (1977)  published  studies  related  to  perceived  attributes  in 
the  middle  1970s.  Keeler  studied  how  to  increase  the  rate  of  adoption  of  a  solar  energy 
product.  He  hypothesized  that  a  better  understanding  of  the  perceived  attributes  held  by 
potential  adopters  in  regard  to  the  product  would  aid  in  the  development  of  effective 
marketing  strategies.  Holloway  studied  the  perceptions  of  high  school  principals  in  regard  to 
an  educational  innovation.  He  found  that  relative  advantage  and  compatibility  were  the 
attributes  most  influential  in  the  principals'  adoption  decisions.  Holloway's  results  are 
similar  to  Moallemian  (1984)  who  found  that  perceptions  of  relative  advantage  and 
trialability  had  significant  influence  upon  the  innovation's  rate  of  adoption. 

Allan  and  Wolf  (1978)  analyzed  the  perceptions  of  100  innovative  educators.  Their 
study  suggests  that  the  attributes  of  Rogers  and  Shoemaker  (1971)  may  not  be  applicable  to 
educational  settings.  Allan  and  Wolf  found  that  only  complexity  had  a  significant  influence 
on  the  likely  adoption  of  educational  innovations.  That  is,  the  less  complex  an  innovation  is 
perceived  to  be,  the  more  likely  that  innovation  is  to  be  adopted.  They  conclude  by  writing: 
"Results  of  this  investigation  .  . .  suggested  innovation  attributes  selected  for  study  provided 
marginal  insight  into  adoption  of  educational  innovations"  (p.  336). 

Rogers,  Daley,  and  Wu  (1982)  studied  the  diffusion  of  home  computers.  They 
determined  that  perceptions  of  relative  advantage,  compatibility,  observability,  and 
trialability  were  important,  ponitive  influences  on  their  subjects'  adoption  decisions. 
Complexity  and  cost  were  found  to  hi1  important,  negative  factors  in  the  adoption  decisions. 
That  is,  the  more  complex  and  cxpetiHive  computers  were  perceived  to  be,  the  less  likely  they 
were  to  be  adopted, 

Ends  (1984)  unci!  a  unique  method  to  examine  how  perceived  attributes  might 


influence  the  rate  of  adoption  of  an  educational  innovation.  While  most  such  studies  have 
employed  questionnaires  and  interviews  in  an  ex  post  facto  design,  Eads  used  written 
descriptions  of  the  innovation's  attributes.  In  all,  thirty  two  sets  of  descriptions  were  used 
to  encompass  all  possible  combinations  of  favorable  and  unfavorable  attributes.  Students, 
teachers,  and  administrators  were  included  in  the  study.  Respondents  were  asked  to  rate 
each  description  set  using  a  semantic  differential  scale.  Eads  discovered  that  compatibility 
was  the  most  influential  attribute  for  students  and  administrators.  No  influential  attribute 
emerged  for  the  teacher  group. 

In  1985,  Dozier  examined  how  the  perceived  attributes  of  electronic  text  might 
influence  the  diffusion  and  adoption  of  that  innovation.  Dozier  writes,  based  on  his  own 
review  of  the  literature,  that  Hprior  research  indicates  that  perceived  attributes  are  powerful 
predictors  of  adoption**  (p.  18).  He  hints  that  electronic  text  could  offer  faculty  members 
"significant  relative  advantages  over  conventional  methods  of  scholarship"  (p.  60).  He  adds 
that  perceived  attributes  of  an  innovation  "particularly  its  relative  advantage,  provide  the 
energy  needed  to  overcome  thresholds  or  barriers  to  adoption"  (p.  -9). 

Two  studies  published  in  1986  (Kehr;  Weinstein)  examined  the  role  that  perceptions 
played  in  the  adoption  and  diffusion  of  educational  innovations.  Kehr  studied  the  diffusion 
of  personal  computers  among  412  full-time  faculty  members  in  college  level  business 
administration  and  teacher  education  programs.  He  found  that  the  groups  shared  similar 
judgments  in  regard  to  the  adoption  of  personal  computers.  Cost  was  the  most  important 
consideration  for  both  groups,  followed  by  the  perceived  attributes  of  the  product.  Weinstein 
studied  the  perceptions  of  civilian  and  military  educators.  The  subjects  for  this  study  were 
244  public  school  teachers  in  Florida  and  397  educators  in  the  U.S.  Marine  Corps.  He  found 
that,  for  both  groups,  the  attributes  of  relative  advantage,  trialability,  compatibility,  and 
observability  were  positively  correlated  with  adoption,  while  the  attribute  of  complexity  was 
negatively  correlated.  Weinstein  concluded  that  perceptions  play  an  important  role  in  the 
adoption  decisions  of  both  military  and  civilian  educators. 

Norris  and  Briers  (1988)  studied  how  agricultural  science  teachers  in  Texas  perceived 
a  proposed  change  in  the  curriculum.  Their  study  is  different  from  the  others  included  in 
this  review  because  Norris  and  Briers  looked  at  perceptions  in  regard  to  the  change  process, 
not  in  regard  to  a  specific  product.  The  sample  for  their  study  was  933  teachers  who  were 
attending  an  agricultural  science  convention.  Norris  and  Briers  found  that  perceptions  of  the 
change  process  were  the  best  predictor  of  a  teacher's  choice  to  adopt  or  reject  the  changes. 
They  also  found,  however,  that  only  16%  of  the  adoption  or  rejection  of  the  changes  could  be 
explained  by  a  teacher's  choice.  Other  factors  accounted  for  most  of  the  variance  in  the 
adoption  or  rejection  decision  process. 

Rogers'  Model  of  Innovation  Diffusion 

The  model  described  by  E.M.  Rogers  is  the  most  comprehensive  model  of  innovation 
diffusion  discovered  in  this  review  of  the  literature.  Rogers  explains  his  theories  in  two 
editions  of  a  book.  These  editions  are  Rogers  (1983)  Diffusion  of  Innovations  (3rd  edition) 
and  Rogers  .ind  Shoemaker  (1971)  Communication  of  Innovations;  A  Cross-Cultural 
Approach  (2nd  edition).  These  two  books,  containing  slightly  different  material,  outline  the 
basic  theories  of  diffusion  and  state  hypotheses  related  to  rate  of  adoption. 

Rogers  and  Shoemaker's  (1971)  Communication  of  Innovations:  A  Cross-Cultural 
Approach  (2nd  edition.)  is  importnnt  for  two  reasons*  The  first  reason,  mentioned  above,  is 
its  large  bibliography  cross-referenced  by  etirh  of  1011  generalizations.  The  second  reason  is 
that  it  is  among  the  first  work*  to  comprehensively  discuss  the  many  factors  which  influence 
an  innovation's  rate  of  adoption. 
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Rogers'  (1983)  Diffusion  of  Innovations  is  important  to  the  current  study  because  it 
presents  an  informative  account  of  the  process  of  innovation  development.  Rogers,  in 
Diffusion  of  Innovations,  is  the  first  author  found  by  this  review  to  state  that  pre  diffusion 
activities,  such  as  development,  play  an  important  role  in  the  diffusion  of  an  innovation. 
The  genera]  premise  of  the  present  study,  that  instructional  developers  can  take  actions  to 
identify  and  account  for  the  perceptions  of  potential  adopters  during  the  development  of  an 
instructional  innovation  and,  as  a  result  enhance  the  subsequent  adoption  of  the  innovation, 
is  based  upon  Rogers*  theory  of  the  innovation  development  process.  Rogers  also  writes 
that  more  research  is  needed  into  the  effects  of  development  and  other  pre-diffusion 
activities  upon  the  subsequent  diffusion  of  an  innovation. 

Methodology 

The  research  design  for  this  study  involved  the  use  of  questionnaires  and  interviews. 
Questionnaires  and  interviews  are  two  of  the  most  commonly- used  methods  of  data 
gathering  in  studies  related  to  diffusion  research  in  the  field  of  education  (Rogers  and 
Shoemaker,  1971).  Instructional  developers  and  potential  adopters  were  asked  to  report 
their  perceptions  of  the  innovation  at  the  time  the  training  modules  were  being  developed  or 
considered  for  adoption. 

Seven  instructional  developers  who  played  a  significant  role  in  the  design  and 
development  of  the  instructional  modules  were  selected  and  interviewed  for  this  study.  The 
seven  developers  who  were  interviewed  for  the  present  study  performed  a  variety  of  tasks 
during  the  development  of  the  three  modules.  An  interview  guide  was  used  to  focus  the 
content  of  the  interviews.  One  developer  interview  took  place  in  a  face  to  face  setting,  the 
other  six  interviews  were  conducted  over  the  telephone. 

Potential  adopters  at  each  of  the  six  forecasting  sites  visited  by  the  researcher  were 
selected  for  the  interviews.  The  forecasting  sites  represented  a  geographically  diverse 
sample  and  included  both  military  and  civilian  settings.  A  total  of  twenty-nine  potential 
adopters  were  interviewed  for  this  study.  An  interview  guide  was  used  to  focus  the  content 
of  all  the  interviews.  All  interviews  were  conducted  in  person  by  the  researcher  at  the 
forecasting  sites. 

All  potential  adopters  at  each  of  the  six  forecasting  sites  were  selected  to  complete  the 
potential  adopters'  questionnaire.  Questionnaire  data  were  collected  from  a  total  of  32 
potential  adopters  of  the  CBL  modules.  While  thirty -two  questionnaires  represent  a 
minority  of  all  the  forecasters  who  work  at  the  six  sites  included  in  this  study,  they 
represent  a  majority  of  the  forecasters  who  had  used  the  modules  at  the  time  for  the  study. 

Results  of  the  Study 
Potential  Adopter  Questionnaire  Data 

Questionnaires  were  distributed  at  six  operational  forecasting  sites  included  in  this 
study.  Completed  questionnaires  were  received  from  five  of  the  sites.  A  total  of  thirty-two 
completed  questionnaires  were  received. 
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Relative  Advantage 


Table  1 

Responses  to  Statements  Related  to  Relative  Advantage 


Statement 

Strongly 
Agree 

Agree 

No 
Opinion 

Disagree 

Strongly 
Disagree 

The  modules  are  a  better  way  to 

cio  on-site  train inc  than  the  u/av 
we  used  to  do  on-site  training. 

11/^ 

/34.4% 

16/ 

4 ./ 

/l2.5% 

1  ./ 

/3.1% 

0 

^/o% 

I  enjoy  on-site  training  a  lot 
more  now  than  I  did  before  we 
had  the  modules. 

6 

/l8.8% 

/6.3% 

7 

1  ^/ 

/3.1% 

/o% 

I  am  really  looking  forward  to 
using  future  modles. 

s ./ 

/l5.6% 

19/ 

/9.4% 

7 ./ 
/21.9% 

0  ./ 

/o% 

1 

/3.1% 

I  think  the  modules  are  effective 
training  tools. 

8 

18/ 

5 /^ 

^/5.6% 

1  ./ 

/3.1% 

0  ./ 

The  modules  are  valuable 
training  tools. 

6  ^/ 

^/is.8% 

21/ 

/>5.7% 

5  ./ 
/15.6% 

0  ./ 

/o% 

0  ./ 

I  think  computer-based  training 
is  the  best  way  for  us  to  receive 
on-site  training. 

5  ./ 

13/ 

9  ./ 
/28.1% 

4  /" 

/T2.5% 

1  ./ 
/f.1% 

Relative  advantage  is  defined  as  the  extent  to  which  an  instructional  innovation  is 
seen  as  being  superior  to,  or  an  improvement  over,  existing  or  competing  products.  The 
questionnaire  contained  six  items  related  to  relative  advantage.  The  responses  to  the  six 
questionnaire  items  referring  to  relative  advantage  are  reported  in  Table  1. 

As  shown  in  Table  1,  responses  to  the  questionnaire  items  on  relative  advantage 
were  mostly  positive.  For  example,  84.4%  of  the  forecasters  reported  that  they  agreed  or 
strongly  agreed  the  modules  "are  a  better  way  to  do  on-site  training  than  the  way  we  used 
to  do  on-site  training."  Also,  75.1%  of  the  forecasters  agreed  or  strongly  agreed  with  the 
statement  "I  enjoy  on-site  training  a  lot  more  now  than  I  did  before  we  had  the  modules." 
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Complexity 

Table  2 

Responses  to  Statements  Related  to  Complexity 


Statement 

Strongly 
Agree 

Agree  1 

No 
Opinion 

Disagree 

Strongly 
Disagree 

The  modules  are  user  friendly. 

is/ 

dy^ 

The  directions  are  adequate  for 
me  to  figure  out  what  I  am 

suoDosedtodo. 

lyS 

y/fi.Wo 

0 y" 

0  y^ 

ySQ>% 

I  had  no  trouble  learning  how  tc 
use  the  modules. 

11/" 

1 yS 

/3.1% 

1 y^ 

y^>.\% 

0  y^ 

yS<S% 

I  knew  when  I  was  supposed  to 
read,  watch  a  video  clip,  or 
answer  a  question. 

13 

by^ 

ZyS* 

o  y^ 

Sometimes  I  felt  lost  and  didn't 
know  how  to  get  to  the  next 

0 

lys 

14  y^ 

5  y^ 

y/\sm 

It  took  me  a  long  time  to 
figure  out  how  to  use  the 

o 

\y^ 

5 

21 y^ 

5  y" 

y^\sm 

The  best  thing  about  the 
modules  is  that  they  are  easy  to 
use. 

3 >^ 

lly^ 

y^53.\% 

11 yS 

\y^ 

ySl.\% 

0  y^ 

y^0% 

Complexity  is  defined  as  the  extent  to  which  an  instructional  innovation  is  seen  as 
easy  to  use  and  user  friendly  by  potential  adopters.  Seven  items  related  to  the  perception  of 
complexity  were  included  on  the  questionnaire.  The  items  refer  primarily  to  the  complexity 
of  the  user  interface  of  the  modules.  The  responses  to  the  seven  questionnaire  items 
referring  to  complexity  are  reported  in  Table  2. 

As  shown  in  Table  2,  responses  to  the  questionnaire  items  on  complexity  were 
mostly  positive.  For  example,  68.8%  of  the  forecasters  reported  that  they  agreed  or  strongly 
agreed  the  modules  "are  user-friendly."  Also,  93.8%  of  the  forecasters  agreed  or  strongly 
agreed  with  the  statement  HI  had  no  trouble  learning  how  to  use  the  modules. w  The 
responses  of  the  survey  items  related  to  complexity  show  that  the  potential  adopters  in  this 
study  perceived  the  innovation  to  have  a  low  degree  of  complexity. 

Compatibility 

Compatibility  is  defined  as  the  extent  to  which  an  instructional  innovation  is  seen  as 
consistent  with  the  existing  values,  beliefs,  enviror^nt,  and  tools  of  potential  adopters. 
Six  items  related  to  compatibility  were  included  in      questionnaire.  The  items  related  to 
compatibility  deal  primarily  with  the  how  the  modules  are  compatible  with  the  forecasters' 
work  schedule  and  content  knowledge  needs.  The  responses  to  the  six  questionnaire  items 
referring  to  compatibility  are  reported  in  Table  3. 

Very  strong  positive  responses  were  given  in  regard  to  the  work  schedule 
compatibility  of  the  modules.  All  32  respondents  agreed  that  breaking  the  modules  into  20 
to  30  minute  segments  was  a  good  way  to  organize  the  modules.  In  another  item  related  to 
work  schedule  compatibility,  66%  of  the  respondents  agreed  that  the  format  of  the  modules 
made  the  modules  easy  to  fit  into  the  forecasters1  schedule  and  workload. 
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All  29  respondents  who  expressed  an  opinion  agreed  that  the  best  thing  about  the 
modules  was  that  they  could  be  used  whenever  the  forecaster  wanted.  Generally  speaking, 
the  responses  to  the  items  related  to  work  schedule  compatibility  were  the  most  strongly 
positive  responses  given  in  regard  to  any  of  the  perceived  attributes. 

Table  3 

Responses  to  Statements  Related  to  Compatibility 


Statement 

Strongly 
Agree 

Agree 

No 
Opinion 

Disagree 

Strongly 
Disagree 

The  modules  cover  topics  I  need 
to  know  about  for  my  job. 

7 

18/ 

^56.395 

3  ./ 

3  ./ 

1  ./ 
/*3.1% 

Breaking  the  modules  into  20  - 
30  minute  segments  is  a  good 
wavtooreanizethem. 

12  ^ 

2<)/ 

0  /^ 

0  /^ 

/o% 

0 

The  modules  provide  realistic 
training  for  my  job. 

3 

(/9.4% 

24/ 

2  ^ 

/6.3% 

3  s*" 

yS\ )A% 

0  /" 

The  format  of  the  modules  make 
it  easy  to  fit  training  into  my 
schedule  at"<  Mtmttoad  

'    6  ^ 

/I8.8% 

15/ 

/o.9% 

4  ^/ 

/l2.5% 

5  /" 

2  /^ 

/6.3% 

I  enjoy  working  with  computers. 

9  /^ 

^^8.1% 

14 /^ 

6 

3  /" 
>/9.4% 

0  /" 

The  best  thing  about  the 
modules  is  that  I  can  use  them 
whenever  I  want. 

24/ 

3  /^ 

o  /" 

^/o% 

^/0% 

Trialability  and  Observability 

Trialability  is  defined  as  the  extent  to  which  potential  adopters  may  test  or 
experiment  with  an  instructional  innovation  prior  to  adoption.  Five  items  related  to 
trialability  were  included  in  the  questionnaire.  Observability  is  defined  as  the  extent  to 
which  the  benefits  and  outcomes  of  an  instructional  innovation  can  be  seen  by  potential 
adopters.  Seven  items  related  to  observability  were  included  in  the  questionnaire. 

Many  potential  adopters  expressed  "No  Opinion"  responses  to  questionnaire  items 
related  to  observability  and  trialability.  For  example,  84%  of  the  respondents  expressed  no 
opinion  when  asked  if  they  had  seen  an  impr,  rement  in  their  own  forecasting  as  a  result  of 
using  the  modules.  Half  of  the  respondents  (16  of  32)  expressed  no  opinion  when  asked  if 
they  had  noticed  that  their  co-workers  seemed  to  enjoy  working  with  the  modules. 

The  short  period  of  time  that  the  modules  had  been  in  use  at  the  work  sites  is  the 
most  likely  reason  for  the  high  "no  opinion"  responses  in  regard  to  the  perception  of 
observability.  At  the  time  the  forecasters  completed  the  questionnaire,  most  of  the  modules 
had  only  been  in  use  at  the  work  sites  for  less  than  a  year.  At  several  sites,  in  fact,  module 
three  had  been  in  use  for  only  a  week  or  two  at  the  time  of  this  study.  As  for  trialability,  the 
forecasters  had  little  say  in  whether  the  modules  were  tried  at  their  worksites.  The 
forecasting  organizations  selected  the  sites  where  the  modules  would  be  used.  As  a  result* 
from  the  forecasters1  view,  there  was  no  perceived  trialability  associated  with  the  modules. 
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Potential  Adopter  Interview  Data 


Twenty-nine  weather  forecasters  from  5  sites  were  interviewed  for  the  present  study. 
A  cross  case  analysis,  grouping  the  responses  of  the  forecasters  in  regard  to  each  of  the  five 
attributes,  is  used  in  this  section  to  summarize  the  interview  data. 

Relative  Advantage 

The  relative  advantage  of  the  computer-based  learning  modules  was  mentioned  by 
all  of  the  twenty- nine  potential  adopters  interviewed  tor  this  study.  Responses  related  to 
relative  advantage  can  be  grouped  into  two  categories  of  advantage.  The  first  category  is 
that  the  modules  are  advantageous  because  they  can  be  used  at  any  time  by  the  forecasters. 
The  second  category  is  that  the  modules  are  advantageous  because  they  are  more 
motivational  and  interesting  than  other  means  of  training.  The  first  category  is  a  merger  of 
the  perceptions  of  compatibility  and  relative  advantage.  Simply  stated,  many  potential 
adopters  felt  the  modules  were  advantageous  because  the  modules  were  compatible  with 
the  work  schedule  of  the  forecasting  office  while  other  forms  of  training  were  not.  The  issue 
of  compatibility  will  be  discussed  in  greater  depth  later  in  this  paper. 

The  second  category  of  relative  advantage  (Motivational  and  Interesting)  was 
commonly  cited  by  potential  adopters  as  an  important  issue.  Whenever  any  of  the 
forecasters  discussed  the  CBL  modules  in  comparison  to  other  methods  of  on-site  training, 
they  always  mentioned  that  the  modules  were  advantageous  because  they  were  more 
motivational  and  more  interesting  than  the  training  alternatives.  The  following  statements, 
taken  from  the  interviews,  are  representative: 

I  enjoy  working  with  the  modules.  They're  more  interesting  than  the  manuals. 
Reading  the  manuals  . . .  you  talk  about  dry,  that's  dry!  But  this  is  interactive, 
more  interesting,  that's  what  I  like  about  it. 

I  think  this  is  a  better  alternative  than  just  sitting  and  reading  a  manual  because 
this  gives  you  a  chance  to  participate.  You  have  the  pictures  and  animations 
whereas  if  you  were  reading  a  manual,  you  would  get  tired  a  lot  quicker  and  get 
bored  at  a  much  quicker  pace. 

I  think  they're  outstanding.  These  modules  are  different,  they're  interesting,  you 
don't  get  bored  like  the  other  types  of  training. 

The  different  types  of  training  we've  had,  they're  boring.  These  modules,  they  have  a 
mix  -  video,  animation,  text.  I  like  the  mix,  it  makes  it  much  more  interesting. 

Complexity 

When  given  the  opportunity  to  respond  to  open-ended  questions  about  the  modules, 
about  half  of  the  potential  adopters  interviewed  for  this  study  mentioned  complexity  as  an 
important  issue.  Two  typical  responses  are  included  here: 

Complexity  is  a  big  issue,  because  when  you1.'  here  on  a  midnight  shift  and 
something  goes  wrong  -  if  you  don't  know  how  to  fix  it,  if  you  hit  a  roadblock,  then 
you  would  tend  to  shut  the  thing  off  and  not  come  back  to  it  for  a  couple  of  weeks. 

'  /  .tat  I  like  most  about  them  is  you  can  sit  down  and  start  using  the 
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training.  You  don't  have  to  know  how  to  use  computers.  They're  very  easy  to 
use  the  first  time  -  you  don't  have  to  learn  how  to  use  the  modules. 

One  of  the  forecasters  interviewed  for  this  study  used  the  open-ended  questions  to 
mention  complexity  as  the  worst  aspect  of  the  modules.  This  forecaster  did  not  have 
substantial  previous  experience  using  computers.  As  a  result,  the  forecaster  found  the 
modules  to  be  somewhat  complicated.  A  sample  of  this  forecaster's  comments  are  included: 

Well,  for  me,  they're  very  complicated.  I'm  not  part  of  the  computer 
generation.  I'm  just  not  comfortable  with  computers.  I'm  still  learning  to 
use  them  so  I  have  that  problem  in  addition  to  learning  the  material. 

Compatibility 

When  given  the  opportunity  to  respond  to  open  ended  questions  about  the  modules, 
all  of  the  twenty-nine  potential  adopters  interviewed  for  this  study  mentioned  compatibility 
as  an  important  issue.  All  twenty- nine  mentioned  that  it  was  important  for  the  modules  to 
be  compatible  with  the  work  schedule  of  the  forecast  office. 

The  weather  is  an  unpredictable  and  capricious  force.  The  workload  in  the 
forecasting  office  is  largely  dependent  upon  the  current  weather  conditions.  When  the 
weather  is  favorable,  forecasters  can  devote  relatively  long  blocks  of  time  to  training.  During 
times  of  severe  weather,  it  maybe  impossible  for  the  forecasters  to  devote  any  time  to 
training.  Several  forecasters  mentioned  that  the  weather  has  a  major  influence  on  how  often 
they  are  able  to  use  the  modules.  Sample  comments  related  to  the  importance  of 
compatibility  w  the  forecasters  are  included  here: 

The  forecaster  has  the  opportunity  when  the  weather  is  benign.  When  it's  kind 
of  quiet  is  a  good  time  to  work  on  the  modules.  He  can  spend  2  or  3  hows 
working  on  them. 

If  you  have  good  weather  (good  weather  evening  shifts,  especially)  it's  not  too 
bad,  you  can  get  back  there  and  use  them.  We  only  get  scheduled  about  4  extra 
shifts  a  month  and  we're  lucky  if  we  keep  those.  So,  if  you're  going  to  use  the 
modules,  the  best  time  is  good  weather  evening  shifts. 

That's  one  of  the  strong  points  of  the  modules,  that  you  can  use  them  in  30  minute 
chunks  uf  time.  Most  days  you  can  keep  an  eye  on  the  weather,  take  30  minutes, 
and  do  the  modules  on  shift.  They're  well  geared  to  the  type  of  job  we  have. 

Fitting  them  into  my  schedule  is  a  big  thing,  the  schedule  prevents  me  from  using 
them  as  much  as  I'd  like.  When  I  get  some  free  time,  to  be  honest  with  you,  I  enjoy 
using  them. 

Trialability  and  Observability 

When  given  the  opportunity  to  respond  to  open  ended  questions  about  the  modules, 
none  of  the  twenty- nine  potential  adopters  interviewed  for  this  study  mentioned  trialability 
as  an  important  issue.  None  of  the  forecasters  mentioned  the  desire  or  the  need  to  use  the 
modules  on  a  limited  basis.  When  asked  specifically  about  trialability,  most  of  the 
forecasters  reported  that  they  were  required  to  use  the  modules  as  part  of  their  work  and 
that  they  were  not  given  the  opportunity  to  experiment  with  the  modules  on  a  limited  basis. 
There  was,  in  effect,  no  trialability  from  the  forecasters'  standpoint 

When  given  the  opportunity  to  respond  to  open  ended  questions  about  the  modules, 
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none  of  the  twenty-nine  potential  adopters  interviewed  for  this  study  mentioned 
observability  as  an  important  issue.  When  asked  specifically  about  the  attribute  of 
observability,  none  of  the  forecasters  reported  that  they  had  seen  any  observable  outcomes 
resulting  from  use  of  the  CBL  mouules. 

The  reason  that  most  forecasters  gave  for  not  observing  any  outcomes  from  the 
modul  was  that  the  modules  had  not  been  in  use  for  very  long.  The  modules  had  been  in 
use  for  less  than  a  year  at  most  sites  included  in  this  study.  Several  forecasters  reported 
that  they  anticipated  that  the  modules  would  eventually  result  in  observable  benefits.  The 
statement  included  here  is  representative  of  most  responses  related  to  the  attribute  of 
observability: 

I  anticipate  seeing  a  change  but  I  haven't  seen  it  yet  and  that's  really  because  we 
just  haven't  gotten  into  it  very  much.  At  this  point,  it  hasn't  really  gotten  into  our 
forecasting  but  it's  just  a  matter  of  time. 

Instructional  Developer  Interview  Data 

Seven  developers  who  contributed  to  the  design  and  production  of  the  three  CBL 
modules  were  interviewed  for  the  present  study.  The  purposes  of  the  interviews  were  to 
determine  the  perceptions  of  instructional  developers  in  regard  to  the  attributes  identified  by 
Rogers  (1983),  to  determine  if  the  developers  were  aware  of  the  perceptions  of  potential 
adopters,  and  to  determine  how  perceptions  affected  the  development  of  the  modules.  Data 
gathered  from  each  of  the  seven  interviews  are  reported  arid  summarized  in  the  following 
section. 

Organizational  Perceptions  vs.  Individual  Perceptions 

Instructional  developers  interviewed  for  this  study  tended  to  discuss  the  modules  in 
terms  of  how  the  modules  fit  into  the  overall  weather  forecasting  organization,  not  in  terms 
of  how  the  modules  affected  the  perceptions  of  individual  forecasters.  This  is  perhaps  the 
most  interesting  finding  of  the  interviews.  In  effect,  the  developers  viewed  the  forecasting 
agencies,  not  the  individual  forecasters,  as  the  potential  adopters  of  the  modules. 

When  asked  about  any  of  the  perceived  attributes  identified  by  Rogers  (1983)  the 
developers  tended  to  discuss  that  attribute  from  an  organizational  perspective.  For 
example,  when  asked  to  describe  the  relative  advantage  of  the  modules,  several  developers 
mentioned  that  the  modules  were  advantageous  because  they  could  be  used  on-site, 
allowing  forecasters  to  stay  at  the  worksite  instead  of  traveling  to  a  training  center*  Having 
the  forecasters  train  on-site  was  advantageous  to  the  organization  for  two  reasons,  according 
to  the  developers.  The  first  is  that  on-site  training  relieved  staffing  and  funding  issues 
associated  with  sending  forecasters  away  for  training.  The  second  advantage  of  on-site 
training  is  that  it  allowed  many  more  forecasters  to  receive  training.  Developer  #1  made  the 
following  continent  when  asked  about  the  relative  advantage  of  the  modules: 

Then1  whh  h  clear  advantage  in  that  the  National  Weather  Service  runs  the  training 
school  and  they  would  have  50  or  fewer  persons  per  year  trained  in  Kansas  City. 
The  need  for  truinihK  f*  Tar  wider  than  that . .  .  there  are  literally  thousands  of 
people  who  ran  benefit  from  the  training. 

Bused  upon  the  seven  developer  interviews,  it  is  possible  to  conclude  that  the 
developers  tended  to  bsso  the  development  of  the  modules  more  closely  on  the  perceptions  of 
the  forecasting  organizations  than  on  the  perceptions  of  the  individual  forecasters.  However, 
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the  developers*  tendency  to  focus  on  organizational  perceptions  is  not  necessarily  improper  or 
worthless.  As  mentioned  above,  the  adoption  situation  examined  in  this  study  was  fairly 
unique  because  there  were  two  sets  of  potential  adopters  -  the  forecasting  agencies  and  the 
operational  forecasters.  The  developers  saw  the  forecasting  agencies  as  the  primary 
potential  adopters  of  the  modules.  As  a  result,  the  developers  concentrated  on  determining 
and  accounting  for  the  perceptions  of  the  organizations. 

Relative  Advantage 

The  instructional  developers  interviewed  for  the  present  study  generally  discussed 
the  attribute  of  relative  advantage  in  regard  to  the  organization  into  which  the  computer- 
based  training  modules  would  be  implemented.  Several  of  the  developers  mentioned  that 
the  modules  were  advantageous  because  they  provided  training  at  the  forecasting  site, 
thereby  eliminating  the  need  to  send  forecaster*  away  for  training.  Providing  training  at  the 
forecasting  site  eliminated  many  staffing  and  funding  problem*  associated  with  sending 
forecasters  away  for  training.  The  reduced  staffing  and  funding  problems  were  a  major 
advantage  of  the  module*  from  the  organization*'  perspective. 

The  developers  commonly  discussed  the  modules  in  terms  of  how  they  would  benefit 
the  organization.  When  asked  how  the  CBL  modules  were  advantageous  to  other  training 
products  available  to  the  forec  nsters.  Developer  #3  discussed  how  the  modules  were 
advantageous  to  the  organization. 

I  visited  forecast  offices!  reviewed  some  of  their  materials.  They've  done  a  variety  of 
things.  They've  had  workshops,  which  are  very  expensive  and  very  hard  on  stalling. 
I  had  some  knowledge  of  other  products,  but  really  not  enough.  There  were  some 
areas  of  overlap  with  other  products  -  we  would  have  been  better  off  looking  more 
into  some  other  products  to  reduce  the  overlap. 

Three  of  the  seven  developers  did  discuss  the  attribute  of  relative  advantage  in 
terms  of  the  how  the  modules  were  advantageous  to  the  forecasters  who  would  use  the 
modules.  Developers  #2  and  #5  mentioned  that  they  thought  forecasters  would  find  the 
modules  more  interesting  and  entertaining  than  other  forms  of  training.  Instructional 
Developer  #2  described  his  feelings  related  to  the  relative  advantage  of  the  modules  this 
way: 

My  feeling  was  that  this  had  the  potential  to  be  a  more  effective  method  of  training, 
but  equally,  I  felt  it  had  the  potential  to  be  a  better  received  method  in  the  field.  I  felt 
people  who  taking  the  modules  would  say  "This  beats  the  classroom1'  or  "This  beats  the 
videotape." 

Developer  #3,  conversely,  mentioned  that  he  felt  parts  of  Module  1  were  boring. 
When  discussing  the  ability  of  the  modules  to  gain  and  maintain  the  users'  interest, 
Developers  #2,  #3,  and  #5  were  discussing  the  attribute  of  relative  advantage  in  terms  that 
closely  resembled  those  used  by  the  forecasters. 

Complexity 

The  complexity  of  the  user  interface  used  in  the  modules  was  an  important 
consideration  for  most  of  the  developers  interviewed  for  this  study.  Developers  #1,  #4,  and 
#7  described  the  process  of  formative  evaluation  through  which  the  developers  gained 
information  about  the  interface  from  the  users.  Changes  were  made  to  the  user  interface 
based,  in  part,  upon  information  gained  from  the  users  during  the  formative  evaluation 
process. 
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When  asked  about  the  complexity  of  the  modules,  many  developers  discussed 
complexity  in  terms  of  the  user  interface.  Developer  #4  worked  on  Module  1  and  played  a 
major  role  in  the  conceptualization  and  development  of  the  user  interface.  Major  elements  of 
the  user  interface  developed  for  Module  1  were  incorporated  into  succeeding  modules.  The 
complexity  of  the  user  interface  was  an  important  consideration  for  Developer  44  during  the 
development  process: 

We  did  lots  of  formative  evaluation  because  of  the  nature  of  the  hypertext  system  we 
were  developing,  the  interface  became  crucially  important.  We  had  done  what  I 
thought  was  a  very  nice  interface.  We  had  done  some  formative  evaluation  and 
tested  it  out  with  the  users. 

Developer  #1  also  mentioned  that  formative  evaluations  were  an  important  source 
of  information  about  the  complexity  of  the  modules: 

Based  on  the  evaluations,  we  changed  drastically  the  presentation  of  some  of  the 
content  that  was  considered  too  complex  or  too  lengthy.  The  nature  of  the  changes 
was  to  simplify  and,  in  some  cases,  remove  things  to  try  to  make  the  content  more 
understandable 

Developer  #7  reported  that  feedback  from  the  potential  adopters  was  used  to  guide 
the  development  of  the  user  interface.  During  development,  the  developer 
perceived  that  the  user  interface  had  a  low  degree  of  complexity.  Developer  #7 
had  this  comment  related  to  the  complexity  of  the  user  interface: 

The  interface  is  probably  one  of  the  best  things  that  we  did.  We  went  back  to 
existing  workstations  and  borrowed  a  lot  of  the  good  concepts.  We  went  through 
a  whole  series  of  Formative  evaluations  with  several  evaluators  coming  in.  That's 
how  the  user  interface  evolved.  It  was  a  very  methodical  process. 

Instructional  Developer  #3  made  an  interesting  point  about  how  modifications 
were  made  to  the  modules  based  on  input  from  the  formative  evaluation  groups: 

In  the  early  stages,  there  are  a  lot  of  changes.  You  get  lots  of  feedback  from  peers, 
subject  matter  experts,  and  formative  review  from  users.  In  the  later  stages,  it  gets 
more  and  more  difficult  to  make  changes  and  the  changes  you  can  make  have 
smaller  and  smaller  impact  on  the  overall  product.  In  the  early  stages,  75  -  90%  of 
the  product  is  changed  based  on  feedback  you  get  from  other  people. 

The  statement  above  points  out  the  need  to  determine  and  account  for  the 
perceptions  of  potential  adopters  early  in  the  development  process.  In  order  to  effectively 
account  for  the  perceptions  of  potential  adopters,  changes  must  be  made  early  in  the  process 
when  changes  are  easier  to  make. 

Compatibility 

Most  of  the  instructional  developers  interviewed  for  this  study  mentioned 
compatibility  as  an  important  issue.  All  of  the  developers  were  aware  of  the  erratic  work 
schedules  at  the  forecasting  sites.  The  developers  generally  understood  that  training  at  the 
forecasting  site  had  to  be  designed  to  allow  for  short  periods  of  use.  Several  developers 
reported  that  their  understanding  of  the  work  schedule  influenced  many  development 
decisions.  Developer  #4,  for  example,  reported  that  computer-based  instruction  was  selected 
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as  the  delivery  medium  for  the  training  in  order  to  provide  maximum  flexibility  to  the 
forecasters.  Developer  #7  reported  that  compatibility  was  one  of  the  main  reasons  for 
selecting  the  computer-based  instructional  medium.  Several  developers  also  reported  that 
tracking  and  bookmarking  functions  were  added  to  the  modules  in  order  to  allow  the 
forecasters  to  leave  and  return  to  the  modules  as  their  schedules  dictated,  thereby  making 
the  modules  more  compatible  with  the  forecasters'  erratic  work  schedule.. 

In  addition  to  their  background  knowledge  of  the  forecasting  site,  instructional 
developers  formed  their  perceptions  of  compatibility  through  a  series  of  formative 
evaluations.  According  to  the  Developer  #1,  the  evaluations  used  forecasters  from  each  of 
the  three  forecasting  agencies.  One  or  two  forecasters  from  each  agency  were  asked  to 
respond  to  mock-ups  and  paper  representations  of  the  modules.  From  the  formative 
evaluations,  the  developing  agency  gained  information  related  to  the  compatibility  of  the 
modules  from  the  users'  viewpoint.  Based  on  input,  changes  were  made  to  the  modules. 
Instructional  Developer  #1  described  the  changes: 

The  decision  to  break  up  the  modules  into  shorter  segments  came  directly  from  the 
tryou's.  When  they  came  in  for  the  tryouts,  they  were  put  off  by  the  long,  detailed, 
laborious  sections  and  they  let  it  be  known  that  they  didn't  like  that  and  were  quick 
to  say  that  they  didn't  believe  other  people  would  go  through  it. 

Developer  #4  also  reported  that  compatibility  was  an  important  consideration  for 
the  development  team: 

Compatibility  was  a  major  reason  for  going  with  the  computer-based  instruction 
approach.  The  forecasters  were  obviously  not  going  to  be  able  to  sit  down  on  a 
consistent  basis  and  use  the  materials.  It  was  going  to  be  a  catch-as-catch-can 
basis.  We  built  tracking  facilities  into  the  design  so  they  could  track  what  parts  of 
the  modules  they  had  accomplished.  The  compatibility  to  a  large  extent  drove  a  lot 
of  the  development  process. 

Compatibility  was  also  an  important  issue  for  Developer  #7.  Included  here  is  a 
representative  response  made  by  Developer  #7  in  regard  to  compatibility: 

Compatibility  was  one  of  the  main  reasons  that  CBL  was  the  medium  of  choice.  You 
couldn't  bring  all  the  forecasters  in  for  training.  The  other  option  that  we've  tried 

many  times  is  to  take  the  experts  out  to  the  field  offices.  You  got  out  to  the  field  office  for  2 
or  3  days,  talk  to  the  forecasters,  then  you  leave.  You  have  to  ask  yourself  'What  kind  of 
impact  did  that  have?1  CBL  was  the  answer  to  that. 

From  the  above  statements,  we  can  see  that  changes  were  made  to  the  module 
based  upon  knowledge  that  developers  gained  related  to  the  perceptions  of  the  potential 
adopters.  We  see  that  potential  adopters,  when  shown  representations  of  the  module, 
perceived  that  the  content  module  was  too  complex.  In  the  second  statement,  we  see  that 
potential  adopters  felt  the  representations  were  laborious  and  overly  detailed.  Based  upon 
that  information,  the  developers  shortened  the  segments  and  made  them  less  detailed. 

Trialability  and  Observability 

Trialability  and  observability  were  not  mentioned  by  any  of  the  seven  instructional 
developers  interviewed  for  the  present  study.  None  of  the  developers  mentioned  that  the 
needs  for  potential  adopters  to  try  the  modules  on  a  limited  basis  or  to  observe  the  benefits 
of  the  modules  wjre  important  considerations  during  the  development  process. 
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Developer  #6  was  one  of  the  few  developers  interviewed  for  this  study  who  discussed 
the  attribute  of  observability  in  detail.  Other  developers,  when  asked  directly  about 
observability,  either  gave  short  responses  or  response**  that  didn't  deal  closely  with 
observability.  The  responses  of  Developer  #6  in  regard  to  the  attribute  of  observability  serve 
as  an  excellent  summary  and  synthesis  of  the  responses  of  the  other  developers  interviewed 
for  this  study.  The  following  response  best  summarizes  the  view  of  all  the  developers  in 
regard  to  the  role  that  observability  played  in  the  development  process: 

[Observability]  is  probably  the  most  difficult  aspect  of  developing  the  modules.  The 
performance  of  the  forecasters  in  the  field  is  so  complex  that  it  is  very  difficult  to  make  any 
kind  of  precise,  direct  correlation  between  a  specific  module  and  a  specific  reduction  in  errors. 
What  we  wanted  is  that  the  effect  will  be  cumulative  over  time.  To  say  that  one  module  had 
a  specific  effect  on  forecasting  would  be  very,  very  difficult. 

Conclusions  and  Recommendations 

Research  Question  One 

What  were  the  perceptions  of  instructional  developers  in  regard  to  relative  advantage, 
complexity,  compatibility,  trialability,  and  observability  during  the  development  of  the 
computer-based  learning  modules? 

Instructional  developers  generally  did  not  relate  the  attributes  defined  by  Rogers  . 
(1983)  specifically  to  the  forecasters.  The  relationship  between  relative  advantage, 
complexity,  compatibility,  trialability,  and  observability  and  the  forecasters'  opinions  of  the 
modules  was  not  commonly  discussed  by  the  instructional  developers  in  this  study.  The 
instructional  developers  in  this  study  more  commonly  discussed  the  process  by  which  the 
modules  were  developed  and  the  implications  of  the  attributes  for  the  organizations  in  which 
the  modules  were  used. 

When  asked  directly  about  the  attributes  of  the  modules,  instructional  developers 
commonly  related  the  attributes  to  organizational  or  systemic  issues  rather  than  to  the 
potential  adopters  of  the  products.  For  example,  when  asked  to  describe  the  relative 
advantage  of  the  modules,  most  of  the  forecasters  in  this  study  stated  that  the  modules 
were  advantageous  because  they  eliminated  the  need  to  take  forecasters  away  from  the 
work  site  for  training. 

Research  Questman  Two 

What  were  the  perceptions  of  potential  adopters  in  regard  to  the  relative  advantage, 
complexity,  compatibility,  trialability,  and  observability  of  the  computer- based  learning 
modules? 

Compatibility,  complexity,  and  relative  advantage  were  important  perceptions  in  the 
forecaster's  decision  to  adopt  or  not  adopt  the  computer-based  learning  modules. 
Trialability  and  observability  did  not  emerge  as  important  perceptions  to  the  potential 
adopters. 

Research  Question  Three 

If  instructional  developers  were  aware  of  the  perceptions  of  potential  adopters,  how  did 
that  awareness  affect  the  development  of  the  modules? 

The  awareness  that  instructional  developers  had  of  the  perceptions  of  potential 
adopters  had  a  mixed  effect  on  the  development  of  the  modules.  The  developers'  awareness 
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of  t  he  percept  ion  of  compatibility  had  a  profound  effect  on  the  development  of  the  modules. 
From  the  very  beginning  of  the  development  process,  instructional  developers  knew  that 
compatibility  was  an  important  issue  to  the  operational  forecasters.  They  knew  this 
because  key  members  of  the  development  team  had  experience  with  the  operational 
forecasting  setting.  The  computer-based  learning  method  was  selected  because  it  would 
allow  the  forecasters  to  use  the  modules  at  their  convenience,  when  time  was  available. 
Also,  as  a  result  of  their  knowledge  of  compatibility,  the  developers  broke  the  modules  down 
into  shorter,  digestible  segments  to  make  the  modules  more  compatible  with  the  forecasters1 
work  schedules. 

During  the  development  process,  instructional  developers  were  aware  of  some  of  the 
perceptions  of  potential  adopters  and  unaware  of  other  perceptions.  Developers  were  aware 
of  the  perceptions  of  potential  adopters  in  regard  to  complexity  and  compatibility.  Most  of 
the  instructional  developers  were  unaware  of  the  perceptions  of  potential  adopters  in  regard 
to  relative  advantage.  Trialability  and  observability  were  not  important  perceptions  for 
either  group. 

Most  of  the  instructional  developers  interviewed  for  this  study  reported  that  they  knew 
the  potential  adopters  had  positive  perceptions  of  the  CBL  nodules  in  regard  to  complexity. 
Instructional  developers  in  this  study  knew  that  the  potential  adopters  perceived  the  user 
interface  and  the  content  of  the  modules  to  have  appropriate  levels  of  complexity. 
Instructional  developers  also  knew  that  the  potential  adopters  would  perceive  the  modules 
as  being  compatible  with  their  needs  and  work  schedule. 

Formative  evaluations  and  personal  experience  were  the  methods  by  which  the 
developers  became  aware  of  the  perceptions  of  the  potential  adopters.  Instructional 
developers  became  aware  of  the  perceptions  of  potential  adopters  in  regard  to  complexity 
through  a  series  of  formative  evaluation  sessions.  Instructional  developers  became  aware  of 
the  perceptions  of  potential  adopters  in  regard  to  compatibility  because  key  members  of  the 
development  team  had  personal  experience  with  the  operational  forecast  setting 

The  developers1  awareness  of  the  perception  of  complexity  had  a  limited  effect  on  the 
development  of  the  modules.  During  the  development  process,  instructional  developers 
determined  the  perceptions  of  forecasters  in  regard  to  complexity  through  the  use  of 
formative  evaluations.  The  knowledge  that  the  developers  had  about  the  potential 
adopters'  perception  of  complexity  had  a  limited  effect  on  the  development  process,  however. 
Based  on  the  knowledge  of  complexity  from  the  users'  viewpoint,  the  developers  simplified  or 
eliminated  some  of  the  content  of  the  modules. 

Research  Question  Four 

How  did  the  perceptions  of  instructional  developers  and  potential  adopters  compare  in 
regard  to  the  relative  advantage,  complexity,  compatibility,  trialability,  and  observability  of 
the  computer-based  learning  modules? 

As  stated  in  Research  Question  One  above,  the  instructional  developers  interviewed 
for  this  study  did  not  commonly  relate  their  design  activities  to  the  perceptions  of  the  end 
users.  The  developers  discussed  their  perceptions  related  to  Rogers'  attributes  only  after 
being  asked  about  them  directly  by  the  researcher.  Kven  then,  the  developers  tended  to 
relate  the  attributes  more  closely  with  the  perceptions  of  the  forecasting  organizations  than 
to  the  individual  forecasters.  The  forecasters,  conversely,  readily  and  consistently  used  the 
attributes  of  compatibility,  complexity,  and  relative  advantage  to  describe  their  reactions  to 
the  modules.  It  is  difficult  to  compare  the  perceptions  of  the  two  groups  lH!cauHO  perceptions 
played  a  much  more  prominent  role  in  the  potential  adopters'  ritneuxMion*  of  the  modules 
than  in  the  discussions  of  the  instructional  developers. 
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Recommendations 


The  findings  resulting  from  the  analysis  of  interviews  with  potential  adopters, 
interviews  with  instructional  developers,  and  the  potential  adopter  questionnaire  lead  to  the 
following  recommendations. 

/.  When  diffusion  is  an  important  consideration,  instructional  developers  should  seek  to 
determine  the  perceptions  of  potential  adopters  in  regard  to  the  attributes  of  complexity, 
compatibility,  and  relative  advantage. 

2.  When  diffusion  is  an  important  consideration,  instructional  developers  should  consider 
the  perceptions  of  potential  adopters  early  in  the  development  process. 

3.  When  diffusion  is  an  important  consideration,  instructional  developers  should  develop 
products  which  create  a  favorable  perception  of  relative  advantage  for  the  potential  adopters. 

4.  When  diffusion  is  an  important  consideration,  instructional  developers  should  develop 
products  which  create  a  favorable  perception  of  compatibility  for  the  potential  adopters. 

5.  When  diffusion  is  an  important  consideration,  instructional  developers  should  develop 
products  which  create  a  favorable  perception  of  complexity  for  the  potential  adopters. 

6.  Instructional  developers  should  use  formative  evaluation  as  a  tool  for  determining  the 
perceptions  of  potential  adopters  in  regard  to  the  attributes  of  complexity,  compatibility, 
relative  advantage. 

Questions  for  Future  Research 

This  study  examined  the  role  of  perceptions  in  the  development  and  adoption  of  an 
instructional  innovation.  The  present  study  provided  insights  into  six  research  questions 
related  to  perceived  attributes.  The  researcher  has  identified  seven  questions  arising  from 
the  present  study  which,  if  studied,  could  provide  valuable  additions  to  the  literature  related 
to  how  perceived  attributes  affect  the  development  and  adoption  of  instructional 
innovations. 

•  7s  there  a  research  methodology  for  the  study  of  adoption  and  diffusion  that  is  more 
efficient  and  practical  than  the  interview  /  questionnaire  recall  methodology? 

•  ■  Vhat  are  the  perceptions  of  other  groups  of  instructional  developers  in  regard  to  the 
relative  advantage,  compatibility,  complexity,  trialability,  and  observability  of  their 

products? 

How  do  the  perceptions  of  potential  adopters  and  instructional  developers  influence 
the  development  and  adoption  of  other  instructional  products? 

•  How  do  the  perceptions  of  instructional  developers  and  ixttential  adopters  compare 
in  situations  where  the  developers  and  adopters  are  separated? 

•  Are  trialability  and  observability  important  perceptions  for  the  potential  adopters  of 
other  instructional  innovations? 
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Do  the  attributes  of  an  innovation  as  identified  by  Rogers  (1983)  adequately 
represent  all  the  attributes  of  instructional  innovations? 

Conclusions 

One  of  the  major  problems  that  instructional  technologists  face  is  that  the  products  of 
instructional  technology  have  not  found  wide  acceptance.  The  underlying  assumption  of  this 
study  is  that  instructional  developers  can  increase  the  acceptance  of  their  products  by 
determining  and  accounting  for  the  perceptions  of  potential  adopters  in  the  development 
process.  / 

The  perceptions  identified  by  E.M.  Rogers  were  important  to  the  potential  adopters  in 
this  study.  All  of  the  potential  adopters  interviewed  in  this  study  used  at  least  one  of  the 
attributes  identified  by  Rogers  to  describe  the  modules.  The  language  of  Rogers  was  very 
much  a  part  of  the  language  of  the  potential  adopters.  The  fact  that  potential  adopters 
used  the  attributes  identified  by  Rogers  to  describe  the  modules  is  an  important  result  of 
this  study.  This  result  shows  that  the  perceived  attributes  of  an  innovation  can  be 
important  considerations  for  those  attempting  to  facilitate  the  adoption  and  diffusion  of 
instructional  innovations.  This  conclusion  agrees  very  closely  with  Wyner's  (1974)  conclusion 
that  the  perceptions  of  those  who  use  an  innovation  can  provide  valuable  data  to  change 
agents. 

Another  major  conclusion  of  this  study  is  that  instructional  developers  most  often 
discussed  the  process  by  which  the  modules  were  developed  or  the  learning  strategies  used 
in  the  modules.  Instructional  developers  very  often  described  the  modules  in  terms  of 
technical  or  instructional  characteristics.  When  asked  directly  about  the  attributes  of  the 
modules,  developers  discussed  the  attributes  in  terms  of  organizational  factors,  not  in  terms 
of  how  they  might  affect  the  individual  potential  adopter. 

Knowledge  of 


&  Content 

Figure  2.  A  model  showing  the  findings  of  the  present  study. 

The  theoretical  model  at  the  beginning  of  this  paper  can  be  modified  to  represent  the 
relationship  that  was  found  to  exist  between  the  instructional  developers,  perceptions,  and 
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potential  adopters  in  the  present  study.  Figure  2  above  is  a  model  representing  that 
relationship.  As  shown  in  Figure  2,  relative  advantage,  compatibility,  and  complexity  were 
important  perceptions  to  the  potential  adopters  in  this  study.  Instructional  developers  had 
no  strongly  held  perceptions  in  regard  to  the  attributes  defined  by  Rogers  but  did  consider 
the  compatibility  and  complexity  of  the  CBL  modules  during  the  development  process.  The 
developers  used  formative  evaluations  and  personal  experience  to  gain  knowledge  of  the 
perceptions  of  potential  adopters  in  regard  to  compatibility  and  complexity.  Based  in  part 
upon  their  knowledge  of  the  potential  adopters  in  regard  to  compatibility  f»nd  complexity, 
the  developers  modified  the  content  and  interface  of  the  modules.  Trialability  and 
observability  were  not  important  perceptions  for  either  group. 

In  conclusion,  the  present  study  sought  to  determine  the  relationship  between 
perceived  attributes,  instructional  developers,  potential  adopters,  and  the  adoption  of  an 
instructional  innovation.  The  relationship  is  a  very  complex  one  and  requires  much 
additional  study.  However,  the  results  of  this  study  indicate  that  the  perceived  attributes 
identified  by  E.  M.  Rogers  do  have  the  potential  to  offer  valuable  insights  to  instructional 
developers  and  change  agents.  The  reader  is  invited  and  challenged  to  formulate  additional 
research  questions,  arising  from  their  own  experiences  and  interpretations  of  the  present 
study,  that  might  further  contribute  to  the  literature  related  to  the  importance  of  perceived 
attributes  in  the  development  and  adoption  of  instructional  innovations. 

The  authors  would  like  to  thank  Dr.  James  Okey  of  the  University  of  Georgia,  for  his 
contributions  to  this  study  and  paper. 
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(Dean's  conference  room.  Four  people  Hit  around  »  table,  shuffling  papers.) 

THE  DEAN:  We're  only  waiting  for  Dewey.  I  hope  he  remembered  the  meeting. 

MICHELE:  I  saw  him  in  the  hall  a  moment  ago  He's  on  his  way,  I  think. 

DEWEY:  (entering  quickly,  and  sitting  down)  I  apologize  for  being  late.  I  stayed  after  class  to 
discuss  some  issues  with  a  few  students. 

HARRY:  Very  commendable. 

THE  DEAN:  Well,  now  that  we're  all  here,  let's  get  started.  The  purpose  of  this 
Appointments  Committee  meeting,  as  you  know,  is  to  decide  whether  or  not  to  recommend  a 
faculty  appointment  for  Linda  Esteban.  We've  talked  about  this  matter  before  at  some 
length,  and  you've  all  had  a  chance  to  look  over  the  file.  The  evaluations,  both  of  Linda's 
teaching  and  of  her  scholarship,  are  very  good.  I'd  have  to  say  that  this  looks  like  quite  a 
clear-cut  decision  to  me.  Do  we  need  further  discussion? 

HARRY:  I'm  afraid  so.  I'd  like  to  raise  an  issue  that's  troubled  me  all  year — but  I  think  it's 
particularly  pertinent  to  this  candidate.  Ixst  me  come  right  to  the  point:  I  have  very  serious 
doubts  about  this  candidate's  scholarship— hers  in  particular,  but  also  about  the  way  we 
evaluate  scholarship  in  general.  Do  we  even  have  any  criteria  or  standards  anymore  for 
distinguishing  good  scholarship  from  bad  scholarship? 

THE  DEAN:  Well,  I've  certainly  assumed  that  we  do. 

HARRY:  I've  assumed  the  same  thing,  but  I'm  beginning  to  wonder.  At  our  last  meeting, 
Dewey  and  Michele  reported  on  Linda's  articles,  and  they  both  said  her  scholarship  was 
excellent.  I  decided  to  look  at  a  couple  of  the  articles,  and  my  reaction  was  different.  In  fact, 
it's  hard  for  me  to  imagine  how  a  reasonable  person  could  say  this  work  is  good.  That's  what 
leads  me  to  wonder  about  our  standards — if  we  have  any. 

Here's  how  I  would  characterize  the  articles.  There  was  a  lot  in  them  that  seemed  calculated 
to  convince  readers  that  the  author  is  a  good,  moral,  compassionate  person  who  is  deeply 
grieved  by  the  injustices  that  Western  civilization  has  inflicted  upon  racial  minorities  and 
women.  Til  concede — although  I  can't  really  know — that  these  expressions  were  perfectly 
sincere.  There  was  also  a  lot  in  the  articles  that  seemed  designed  to  establish  the  author's 
credentials  as  a  member  of  a  class  that  has  been  victimized  or  oppressed.  All  of  this 
presumably  relates  to  the  question  of  'Voice" — the  author  assures  us  that  she  speaks  with  the 
proper  "voice." 

I'll  concede  that  much  too;  the  author  has  the  right  "voice"  (whatever  that  means).  But  it  still 
seems  to  me  that  a  scholar  has  an  obligation  to  use  her  voice  to  say  something  coherent  and 
significant.  And  that's  where  the  articles  fail.  They  have  the  right  "voice,"  maybe;  and  they 
use  the  right  postmodern/critical/feminist  vocabulary.  But  they  just  don't  say  anything 
significant. 

MICHELE:  Getting  a  little  carried  away,  aren't  we,  Harry? 

HARRY:  Okay,  I've  overstated  my  point.  I  agree  that  the  articles  make  a  number  of  claims 
that,  if  true,  would  be  very  significant.  The  basic  problem  is  that  there  is  almost  no  effort  to 
back  up  those  claims  with  anything  like  evidence  or  careful  analysis.  There  are  also  what  I 
take  to  be  some  philosophical  positions  implicit  in  the  articles;  but  once  again,  far  from 
articulating  those  positions  and  then  making  serious  arguments  for  them,  the  author  doesn't 


appear  to  understand  what  they  are  or  where  they  come  from.  In  the  end,  the  articles  seem 
more  like  ventilation  than  reasoned  argument.  It's  as  if  she's  saying,  "Look,  I'm  a  sensitive 
person,  and  I've  been  oppressed,  and  here's  my  opinion" — and  we're  supposed  to  just  accept 
it.  But  if  you  forget  for  a  moment  about  the  'Voice"  and  the  indignation  and  just  look  at  the 
substance — the  arguments  made  and  the  way  they  are  or  aren't  supported — these  articles 
would  barely  get  "Bs"  in  a  freshman  writing  course.  I  ought  to  know;  I  used  to  teach 
freshman  writing  classes  before  I  went  to  Florida  State. 

MICHELE:  Frankly,  Harry,  it  strikes  me  that  your  own  speech  shows  that  these  articles 
have  more  to  them  than  you  give  them  credit  for.  In  fact,  you're  acting  as  Exhibit  "A"  for 
Linda's  thesis  in  her  most  recent  article.  You're  applying  traditional  white  male  criteria  to  an 
author  who  takes  a  different  perspective,  and  as  a  result  you're  dismissing  her  without  really 
even  hearing  what  she's  trying  to  say. 

HARRY:  I  knew  someone  would  say  that;  in  fact,  that's  why  I've  kept  quiet  about  these 
issues  up  until  now.  Anyone  can  use  that  rhetorical  trick.  Just  call  something  "white  male," 
and  it  doesn't  matter  what  it  is — traditional  classrooms,  teachers,  instructional-design 
models,  why  not  computers? — you  think  you've  discredited  it.  For  some  reason — I  can't  quite 
figure  it  out — no  one  ever  seems  to  remember  that  Marx,  Nietzsche,  Freud,  and  Rousseau 
were  all  white  males  too. 

ARIEL:  So  were  Gadamer,  Habermas,  Derrida,  and  Foucault. 

HARRY:  No  offense  intended,  but  I  think  that  argument  is  just  a  cover  for  intellectual 
laziness.  And  I'm  not  buying  it  anymore.  When  I  say  that  these  articles  are  vacuous,  I'm  not 
applying  "white  male"  criteria.  I'm  judging  them  by  their  merit — or  by  their  lack  of  merit. 

MICHELE:  A  kinder,  gentler  term  for  the  same  thing.  "Merit"  according  to  white  male 
standards. 

HARRY:  That's  not  true.  When  someone  says  that  scholars  ought  to  back  up  claims  with 
evidence,  or  that  they  ought  to  reason  and  analyze  instead  of  just  emoting,  or  that  they  ought 
to  follow  through  on  the  logic  of  what  they've  said  without  contradicting  themselves,  those 
are  not  in  any  sense  "white  male"  standards.  They  are  simply  objective  standards  for  good 
scholarship,  no  matter  who's  doing  it. 

DEWEY:  Let  me  jump  in  here,  if  I  may.  This  discussion  has  all  been  very  interesting.  And  I 
think  Harry's  last  statement  is  especially  revealing;  it's  also  ironic,  in  view  of  his  earlier 
complaint  about  Linda  Esteban's  lack  of  philosophical  sophistication  or  self-awareness. 
Because  it  seems  that  Harry  has  his  own  tacit  philosophical  assumptions  that  he  may  not 
have  examined  very  carefully.  He  has  a  sort  of  metaphysics  and  epistemology  that  we  might 
describe  as  "realist." 

HARRY:  I  don't  accept  that.  As  a  matter  of  fact,  I  don't  even  know  anything  about 
philosophy,  except  for  the  little — very  little,  I  assure  you— that  I  remember  from  the  survey 
philosophy  course  I  took  as  an  undergraduate.  So  I  don't  have  any  particular  philosophical 
views  at  all.  We  instructional  designers  manage  quite  nicely  without  any,  thank  you. 

DEWEY:  You're  being  too  modest,  Harry.  Actually  you're  more  of  a  philosopher  than  you 
realize. 

HARRY:  Me — a  philosopher?  Now  that's  really  hitting  below  the  belt. 

MICHELE:  I  was  under  the  impression  that  the  purpose  of  this  meeting  was  to  discuss 
whether  Linda  Esteban  should  be  appointed — not  whether  Harry  is  a  closet  philosopher. 
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Perhaps  we  could  Ret  back  to  business? 

HARRY:  And  just  leave  Dewey  s  slanderous  accusation  unexplained  and  unanswered? 
ARIEL:  I'm  interested.  What  did  you  mean!  Dewey? 
DEWEY:  Just  this— and  I'll  be  succinct  about  it,  Michele. 

It's  true  that  Harry  hasn't  explicitly  endorsed  any  metaphysical  position,  of  course.  But  think 
about  what's  assumed  in  his  notions  of  "merit,"  "reasoned  argument/'  and  especially  his 
ir  sistence  on  supporting  assertions  with  "evidence."  Harry  has  been  less  than  clear  about 
what  would  count  as  evidence,  but  it  seems  pretty  obvious  that  things  like  "Voice"  and 
personal  perspective  aren't  evidence.  Those  first  person  aspects  are  entirely  distinct  from 
what  Harry  calls  the  "substance"  of  the  articles,  and  they  don't  count  towards  establishing 
the  "truth"  of  what  an  author  like  Linda  asserts.  Other  things — like  controlled  experiments 
with  lots  of  statistics — apparently  do  count.  But  what  are  the  background  assumptions  that 
would  lead  Harry  to  take  a  position  like  this? 

HARRY:  I  don't  know.  Since  Dewey  seems  to  understand  my  philosophical  views  better  than 
I  do,  maybe  hell  explain  to  me  what  they  are. 

DEWEY:  I'll  be  happy  to  try.  Harry  assumes  that  there  is  some  sort  of  reality  "out  there";  it 
exists  independent  of  the  person  who  is  talking  about  it.  And  a  statement  is  "true"  if  it 
accurately  describes,  or  corresponds  to,  that  reality.  That's  why  Harry  can  think  that  his 
standards  of  merit  are  "objective."  The  scholar  is  not  trying  to  impose  anything,  any  values  or 
goals  or  biases,  onto  reality;  she  is  just  trying  to  describe  it,  And  she  shows  that  she  has 
given  a  correct  description  by  supporting  her  statements  with  evidence,  with  careful  logical 
analysis,  and  so  forth.  Just  "emot  ing,"  or  "ventilating,"  aren't  helpful;  they  tell  us  nothing  at 
all  about  the  "object"  being  described,  whatever  that  is;  they  only  tell  us  about  the  subject,  or 
the  speaker.  So  as  far  as  objective  description  goes,  these  kinds  of  expression  only  clutter 
things  up.  Am  I  getting  your  position  right,  Harry? 

HARRY:  I  don't  want  to  be  pinned  down  to  any  philosophical  position,  particularly  one 
supplied  by  you,  Dewey.  But  in  this  case,  I'll  have  to  admit  that  what  you  said  sounded 
pretty  much  correct.  That's  just  common  sense;  everyone  acknowledges  that  a  "true" 
statement  iB  one  that  accurately  describes  what  it's  talking  about. 

DEWEY:  No,  not  everyone  does.  But  let  me  continue.  Harry's  unreflective  metaphysical  and 
epistemological  assumptions — which  I  think  we  can  refer  to  collectively  as  "realism" — 
express  one  possible  philosophical  position.  And  of  course  that  position,  in  its  various 
versions,  has  some  venerable  names  associated  with  it:  Descartes,  Bacon,  Locke.  It  probably 
also  expresses  the  view  that  ordinary  people  typically  take,  at  least  when  they  haven't  given 
much  thought  to  the  philosophical  issues  involved.  And  a  realist  position  can  support  certain 
criteria  of  "merit"  for  judging  scholarship.  That's  the  good  news  (for  Harry). 

But  here's  the  bad  news.  The  realist  understanding  of  truth  is  only  one  possible  philosophical 
position — not  the  only  one.  And  I  would  say  that  today,  very  few  philosophers  or  academics 
interested  in  these  issues  accept  the  "realist"  position.  In  fact,  I  would  go  further  and  suggest 
that  the  "realist"  position  has  been  thoroughly  discredited.  So  it  turns  out  that  while  Harry 
criticizes  Linda  Esteban  for,  among  other  things,  not  understanding  her  philosophical 
presuppositions,  his  own  criticisms  arise  out  of  philosophical  assumptions  that  he  doesn't 
seem  to  have  examined  —and  that  in  fact  are  thoroughly  outdated. 

THE  DEAN:  Well,  Harry,  I  guess  you've  been  put  in  your  place. 

HARRY:  1  don't  think  so,  Dewey  hasn't  shown  what's  wrong  with  the  standards  that  I  (and, 


until  recently,  all  of  us)  want  to  apply — with  standards  that  suggest  that  you  should  back  up 
your  assertions  with  evidence*  with  careful  logical  analysis,  and  so  forth.  All  he's  done  is 
attribute  to  me  a  particular  philosophical  position  and  then  report  that  the  position  isn't 
popular  with  philosophers  at  the  moment.  As  far  as  I'm  concerned,  that's  their  problem — and 
Dewey's.  Unless,  that  in,  ho  can  show  me  some  plausible  philosophical  position  other  than 
tin*  "realiHt"  one  and  explain  how  under  that  alternative  position  something  like  supporting 
assertions  with  evidence  isn't  a  valid  standard. 

AU1ICL:  That  sounds  like  a  challenge. 

MK'IIKLE:  Yes,  I'm  afraid  it  does.  And  of  course  it  would  be  unmanly  to  let  a  challenge  go 
unanswered.  So  if  you'll  excuse  me,  I'll  be  back  in  a  moment.  It's  clear  I'm  going  to  need 
Home  caffeine. 

TI IK  I) KAN:  There's  some  soda  in  the  fridge.  Help  yourself. 

MUHl  KMC:  Tin  very  grateful.  I'm  sure  I  wouldn't  have  wanted  to  miss  out  on  any  of  this 
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DKWKY:  Well,  I  can't  accept  the  challenge  in  quite  the  terms  that  Harry  used  just  now,  but 
I'll  try  to  deal  with  the  problem  he  raises.  Let's  start  with  Harry's  notion  of  what  it  means 
for  a  statement  to  be  "true"  and  see  where  things  lead.  Harry  thinks  that  a  statement  is 
"true"  if  it  accurately  describes — or  agrees  with,  or  corresponds  to — some  independent 
reality.  Now  if  we  think  about  it,  isn't  it  clear  that  this  can't  be  what  "truth"  means? 

HARRY:  I  don't  see  why  it  can't. 

DEWEY:  For  a  number  of  reasons.  But  the  main  problem,  I  think,  is  that  if  the  realist 
notion  of  truth  were  the  correct  one,  we'd  never  be  able  to  know  whether  a  statement  is  really 
true.  And  a  conception  of  "truth"  that  leaves  us  completely  unable  to  know  whether  any 
given  statement  is  true  wouldn't  be  too  helpful,  would  it? 

ARIEL:  I'm  not  following.  Why  does  a  realist  definition  of  truth  leave  us  unable  to  know 
whether  a  given  statement  is  true? 

DEWEY:  Mainly  because' of  what  you  might  call  the  "no  getting  out  of  your  own  skin" 
problem.  Let's  take  a  very  t  imple  example — the  kind  of  example,  in  fact,  that's  most 
favorable  to  Harry's  view.  Suppose  I  say  something  like  "The  chair  in  the  corner  is  brown." 
By  Harry's  account,  my  statement  is  true  if,  independent  of  me,  there  is  in  fact  a  chair  in  the 
corner  and  it  is  in  fact  brown.  (We'll  leave  aside  for  now  the  additional  problem  that  the  very 
ideas  of  "chair"  and  "brown"  are  human  constructs.)  Now,  in  order  to  verify  those  facts — i.e., 
that  there  is  a  chair  "out  there"  and  that  it  is  brown — I'd  need  to  be  able  somehow  to  step 
outside  myself,  set  my  statement  about  the  chair  alongside  the  chair  itself  (or  whatever  there 
may  be  "out  there"),  and  see  if  they  match.  Obviously,  I  can  never  do  that,  Like  it  or  not,  I'll 
always  be  stuck  inside  myself. 

ARIEL:  That's  certainly  true.  But  you  could,  if  you  were  really  worried  about  thin,  link  me  to 
check  out  the  chair  for  you.  I'm  outside  your  skin.  I  could  verify  your  statement  for  you. 

DEWEY:  We  could  do  that,  yes.  But  notice  that  your  suggestion  dotmn't  solve  my  problem. 
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If  you  tell  me  that  the  chair  in  the  corner  is  brown,  then  I  would  have  two  things  to  work 
with:  my  own  initial  perception,  and  now  your  report  as  well.  What  I  wouldn't  have — never 
could  have — is  the  ability  to  match  my  perception  and  my  statement  of  my  perception  against 
the  chair  itself.  And  you  can't  do  that  for  me,  because  you  can't  get  outside  your  skin  either. 

ARIEL:  So  then  we're  right  back  where  we  started? 

DEWEY:  Not  quite.  I  do  have  something  else  to  check  my  statement  against — Your 
statement.  Now  suppose  we  bring  ten  more  people  in  here,  and  they  all  say,  "Yes,  the  chair  in 
the  corner  is  brown."  We'd  be  pretty  confident  then  in  saying  that  my  initial  statement — "The 
chair  in  the  corner  is  brown" — is  "true."  But  here's  the  crucial  point:  We  would  say  the 
statement  is  true  not  because  we  (individually  or  collectively)  could  match  the  statement 
against  the  chair  itself.  What  we  match  up,  rather,  is  our  statements.  In  other  words,  what 
we  would  really  mean  in  saying  that  my  statement  is  "true"  is  that  it  fits  well — coheres,  we 
might  say  -  with  other  people's  statements  and  beliefs,  not  that  it  matches  or  agrees  with  the 
object  itself.  Truth,  we  might  say,  is  a  product  of  dialogue — a  sharing  and  comparing  of 
statement!*  and  beliefs— and  the  working  criterion  of  truth  is  dialogical  coherence,  not 
objective  correspondence. 

ARIEL:  So  instead  of  a  "realist"  theory  of  truth,  we  have  a  "dialogical  coherence"  theory.  Is 
that  what  you're  saying? 

DEWEY:  That's  about  right.  Of  course,  what  I've  given  here  is  only  the  skeleton  of  a  theory. 
Different  people  would  flesh  it  out  in  lo';s  of  different  ways.  But  I  think  it's  fair  to  say  that 
most  philosophers  today,  and  most  educational  theorists  who  have  thought  seriously  about 
these  issues,  subscribe  to  some  version  of  a  dialogical  coherence  theory.  The  common  element 
lies  in  the  recognition  that  we  don't  discover  truth  by  reading  it  off  of  an  objective  reality; 
instead  we  construct  truth — through  dialogue. 

HARRY:  And  it  follows,  I  suppose,  that  in  a  dialogical  coherence  theory  of  truth,  standards 
like  "Support  your  assertions  with  evidence"  have  to  be  discarded? 

DEWEY:  Not  at  all.  That's  what  always  happens;  someone  hears  that  a  "realist"  theory  of 
truth  has  been  criticized,  and  they  immediately  attribute  all  kinds  of  dire  consequences  and 
outlandish  implications  to  an  alternative  position.  Actually,  standards  like  Harry's  would 
still  have  their  place  in  a  dialogical  coherence  theory.  But  their  significance  would  be  a  little 
different. 

HARRY:  Youll  elaborate,  I'm  sure. 

DEWEY:  Of  course.  Once  we  understand  that  truth  is  a  product  of  dialogue,  then  we  can  also 
see  that  within  any  particular  community  that  engages  in  dialogue,  there  will  be  certain 
practices  that  govern  the  making  and  acceptance  of  statements.  These  practices  might  vary 
from  one  community  to  another.  In  one  community,  the  practice  might  be  that  statements 
ought  to  be  consistent  with  some  sacred  scripture.  In  another  community — one  heavily 
influenced  by  the  methodologies  of  empirical  science,  for  instance — practice  might  demand 
that  statements  be  supported  by  evidence  gained  through  Bensory  perception.  Since  these 
practices  determine  the  kinds  of  statements  that  can  be  made  and  accepted,  they  will 
naturally  determine  what,  for  that  community,  can  be  accepted  as  "true."  In  our  own 
community,  it  happens  that  we  have  developed  a  practice  that  expects  people  to  make 
statements  that  can  be  supported  with  "evidence," — usually  empirical — although  I'd  suggest 
that  we're  not  too  clear  about  what  counts  as  evidence.  Still,  the  standard  Harry  invokes  is 
perfectly  understandable  and  even,  in  the  sense  I've  been  describing,  legitimate. 

But  there  is  a  lot  more  to  be  said  now.  In  the  first  place,  the  standards  accepted  by  a  given 
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dialogical  community  reflect  the  practice*  of  that  community.  So  it  would  be  misleading  to 
assume  that  these  practices  or  standards  lire  "neutral"  or  "objective1*  in  any  larger  or 
universal  sense. 

Also,  the  practices  of  u  community  wilt  make  it  difficult  for  someone  coming  from  a  different 
community  governed  by  different  prai'ticoH  to  Hpcuk,  or  to  be  understood,  in  the  first 
community.  In  that  sonse,  tht1  standards  of  (ruth  that  are  grounded  in  those  practices  will 
operate  to  exclude?.  So  when  Mirhelc  rifcr*  tu  Harry's  standards  as  "white  male"  standards, 
she  has  a  valid  point.  I  larry  dm'Hit't  need  to  t  akt*  offense;  Michele  isn't  saying  that  Harry's 
standards  are  wrong  in  any  ultimate  sense,  Indeed,  she  couldn't  say  that,  because  to  say  it 
would  presuppose  the  ram*  kind  of  tJiitemitrit'  ohjectivist,  universalist  assumptions  that 
Harry  himself  is  implicitly  usiiiK.  Hut  Harry's  standards  are  based  on  practices  that  happen 
to  have  prevailed  in  a  dialogical  cumin  unity  composed  mainly  of  white  males. 

Harry:  So  everything  relative?  How  then  can  we  guy  anything  is  really  true? 

Dewey:  Oh,  I  can  say  it  all  right.  It's  true  that  we're  all  sitting  here  talking.  I  believe  it's 
really  true.  But  the  meaning  of 't  rue'  means  Hornet hing  a  little  different  to  me  than  it  means 
to  you.  If  you  mean  'True"— with  a  capital  T— then  I'm  not  sure  anyone's  got  a  handle  on 
that  completely. 

Another  thing — Once  standards  are  undcrHtood  to  he  grounded  in  practices  that  are 
contingent— by  that  I  mean  that  the  practices  might  be  different,  and  in  fact  are  different, 
from  one  community  to  another — then  it  also  follows  that  the  practices  and  standards 
become  eligible  for  reexamination  and,  possibly,  for  change.  And  something  else  follows  that 
is  vital  to  the  general  question  Harry  raised  about  evaluating  scholarship.  If  we  understand 
that  truth  results  from  dialogue,  from  talking  together,  not  from  comparing  statements  with 
some  independent  reality,  then  we  will  naturally  be  much  more  concerned  with  the 
participants  in  the  conversation.  Who  gets  to  participate?  What  kind  of  people  are  doing  the 
talking?  How  do  these  people  perceive  and  describe  things?  What  kind  of  language,  or  what 
kinds  of  conceptual  schemes,  do  they  use?  I  could  put  it  differently:  Once  we  realize  that 
truth  is  constructed,  we  naturally  become  interested— primarily  interested,  I  might  even 
say— in  who's  doing  the  constructing.  So  all  of  these  "first  person"  questions  about  'Voice" 
and  who  the  speaker  is —  questions  that  in  Harry's  realist  view  can  be  quickly  brushed  aside 
because  they  don't  bear  directly  on  the  objective  "truth"  of  what  is  said— come  to  be  vitally 
significant. 

THE  DEAN:  Sof  Harry,  are  you  ready  to  concede  the  point? 

HARRY:  Very  amusing.  No,  I'm  not  convinced.  I  think  Dewey's  just  given  us  a  fine  example 
of  sophistry  to  try  to  get  us  to  relinquish  what  we  all  know.  But  I  can't  articulate  off  the  top 
of  my  head  just  what  the  mistake  in  Dewey's  argument  is;  I'm  not  a  philosopher,  after  all. 


Ill 


THE  DEAN:  Well,  then,  perhaps  we  can  get  back  to  business  and  consider  the  question  of 
Linda  Esteban's  appointment.  Any  further  discussion? 

ARIEL:  Actually,  before  we  discuss  the  specifics  of  Linda's  candidacy,  I'd  like  to  explore 
Dewey's  argument  a  little  further.  Would  that  be  appropriate? 

THE  DEAN:  Well,  you  realize  this  is  supposed  to  be  an  Appointments  Committee  meeting, 
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not  a  gathering  of  the  Socratic  society. 

MICHELE:  Couldn't  you  and  Dewey  resume  this  discussion  later? 

ARIEL:  We  could,  and  if  no  one  else  wants  to  continue  this  discussion,  I'll  just  drop  it.  But 
Dewey  did  tell  us  that  these  philosophical  questions  bear  directly  on  the  standards  that  wo 
should  apply  in  evaluating  scholarship.  If  that's  right,  and  since  the  issue  has  been  raised,  it 
seems  to  me  that  we  ought  to  pursue  this  a  little  further. 

THE  DEAN:  I'll  defer  to  the  other  members  of  the  Committee,  But  remember  that  some  of  uh 
may  have  to  leave.  The  more  time  we  spend  on  metaphysics,  the  less  likely  it  is  that  we'll  be 
able  to  make  a  decision  on  Linda  Esteban  today.  And  if  we  don't  move  quickly,  were  likely  to 
lose  her. 

DEWEY:  I  share  the  Dean's  concerns.  But  I'd  hate  to  truncate  discussion  if  some  people 
think  it's  relevant  to  our  decision. 

HARRY:  I'm  willing  to  listen  a  little  longer.  Especially  if  this  discussion  will  spare  me  from 
reading  all  the  articles  that  ought  to  be  in  philosophy  journals  but  have  somehow  gotten  into 
the  Ed  Tech  journals. 

THE  DEAN:  I  wouldn't  count  on  it.  But  it  sounds  like  you  have  leave  to  go  ahead,  Ariel.  Let  's 
just  try  to  be  as  efficient  as  possible. 

ARIEL:  I  will.  This  is  my  difficulty:  I  can  accept  some  of  what  Dewey  said,  but  not  his 
relativism  or  his  total  rejection  of  truth  as  correspondence.  He  insists  that  "truth"  is 
constructed;  and  in  one  sense  that  seems  right.  After  all,  truths  have  to  be  expressed  in 
language,  and  language  is  socially  constructed.  (Although  I  have  to  admit  that  "constructed" 
doesn't  seem  to  be  quite  the  right  word;  it  implies  that  we  have  more  conscious  control  over 
the  process  than  we  actually  do,  I  think.)  And  our  statements  about  what  we  think  is  true 
will  naturally  use  the  concepts  and  categories  and  theories  that  are  available  to  us,  and  those 
are  also  constructed.  If  that's  all  that  Dewey  means,  then  I  could  probably  just  agree  with 
him. 

Dewey:  What  1  mean  is  that  the  meaning  of  things  is  within  us,  not  within  the  things 
themselves. 

ARIEL:  Well  Dewey,  I  still  don't  see  why  you  have  to  reject  the  realist  view  of  truth.  It  seems 
to  me  that  I  can  admit  that  truth  is  "constructed"  in  the  ways  I've  mentioned  and  still 
maintain  that  a  statement  is  true  if  it  accurately  describes  the  thing  it's  talking  about, 

DEWEY:  But  it's  like  an  endless  circle,  trying  to  figure  out  what  'accurately  describes' 
means  in  any  purely  objective  sense. 

ARIEL:  Let  me  give  an  example:  Suppose  Dewey  says,  "The  Dean  weighs  180  pounds." 

THE  DEAN:  Now  I'm  getting  interested.  And  if  Dewey  can  come  up  with  a  theory  that  makes 
that  statement  true,  I'll  see  to  it  that  he  gets  an  instant  promotion. 

HARRY:  I'm  afraid  that  would  be  beyond  even  Dewey's  sophistical  powers.  Philosophizing  is 
never  going  to  replace  dieting. 

ARIEL:  It's  a  purely  hypothetical  example.  My  point  is  this.  In  some  senses,  the  truth  of 
Dewey's  statement — we'll  fantasize  that  it  is  true — might  be  said  to  be  "constructed."  After 
all,  the  statement  doesn't  even  mean  anything  except  in  a  language  that  is  socially 


constructed.  And  it  uses  concepts—  'pounds/  for  instance — that  are  socially  constructed. 
'Pounds'  as  a  term  of  meaHurement  is  a  human  invention;  we  might  just  as  easily  talk  about 
'kilos'  or  something  else.  The  Dean  doesn't  come  with  a  number  of  pounds  written  on  him. 


DEWEY:  I'm  with  you  so  far, 

ARIEL:  My  point  is  that  none  of  this  seems  to  mean  that  we  have  to  abandon  a  realist  notion 
of  "truth."  It  seems  more  sensible  to  say  that  what  a  statement  means  is  a  matter  of 
convention  or  social  construction;  but  whether  the  statement  is  true  still  depends  upon 
whether  it  accurately  describes  or  agrees  with  its  object.  In  this  case,  unfortunately,  it 
probably  doesn't. 

But  Dewey  apparently  believes  that  because  in  certain  senses  all  true  statements  are 
"constructed,"  we  therefore  have  to  give  up  the  idea  that  truth  depends  upon  agreement  of  a 
statement  with  the  "reality"  it  describes.  Instead,  a  statement  is  true  if  it  coheres  with — I'm 
not  too  clear  on  just  what  that  means — other  statements  and  beliefs  commonly  expressed  and 
accepted  in  the  dialogue  of  a  given  community.  Isn't  that  more  or  less  what  you  were  saying? 

DEWEY:  More  or  less.  But  I  don't  think  you  can  take  the  moaning  out  of  a  statement  and 
judge  it  purely  on  its  'truth'  value.  What  does  it  mean  for  something  to  be  true  if  the 
meaning  of  it  is  unclear?  You're  using  'truth'  in  a  very  narrow  sense  that  robs  it  of  its 
richness;  I'm  not  sure  what  you've  got  left  after  meaning  is  separated  out. 

ARIEL:  Well,  it  seems  to  me  that  your  understanding  of  truth  clashes  in  all  sorts  of  ways 
with  what  everyone  understands  'truth'  to  mean. 

DEWEY:  It  very  well  may.  But  can  you  be  more  concrete? 

ARIEL:  For  instance,  most  people  would  think,  to  use  your  example,  that  you  can  say  "The 
chair  in  the  corner  is  brown"  and  your  statement  is  true  or  false  regardless  of  whether 
anyone  else  comes  into  the  room  and  agrees.  Suppose  you  yourself  placed  the  chair  in  the 
corner  on  Monday  morning  and  observed  that  the  chair  in  the  corner  was  brown,  and  then 
the  building  burned  down  so  that  no  one  else  ever  saw  it.  According  to  your  theory  of 
"dialogical  coherence/  it  seems,  your  observation  would  just  be  hanging  out  there;  it  would 
never  be  true  or  false  because  there  would  never  be  any  dialogue  to  confirm  or  disconfirm  it. 
Hut  that  seems  crazy;  no  one  believes  that. 

DEWEY:  But  whether  we  like  it  or  not,  we  are  in  the  world,  and  in  conversation  with  people 
past  and  present.  Our  culture  is  the  context  within  which  we  perceive  chairs  and  everything 
else  in  this  world.  I  may  never  have  talked  to  anyone  about  the  chair,  but  my  actions,  beliefs, 
and  thoughts  about  it  are  all  conditioned  by  my  participation  in  the  world. 

ARIEL:  OK,  I'm  starting  to  see  your  point.  But  let's  take  a  different  example  that's  close  at 
hand.  The  Dean  says,  'There's  pop  in  the  refrigerator."  What  that  statement  means  depends, 
I  agree,  upon  conventions  of  language  and  and  practice  that  are  socially  constructed.  But 
does  the  truth  of  the  statement  depend  upon  coherence  within  a  cultural  context?  I  don't 
think  so.  It  would  be  just  as  "coherent"  to  say,  "There  isn't  any  popin  the  refrigerator."  The 
truth  of  the  statement  depends  on  whether  then?  is  in  fact  a  fridge  "out  there,"  and  whether 
it  in  fact  has  pop  in  it. 

Or  suppose  I  say,  "The  Yankees  will  win  the  pennant  this  year,"  and  nobody  else  agrees. 
Does  that  mean  my  statement  is  false?  Even  the  people  who  think  the  Yankees  don't  have  a 
chance  wouldn't  say  that.  They  would  say  that  my  statement  is  probably  false,  but  that  we 
won't  really  know  for  sure  until  we  see  who  actually  does  win  the  pennant.  And  even  then,  if 
my  statement  turns  out  to  he  false,  it  won't  be  because  other  people  disagreed  with  it.  It  will 
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be  because  the  Yankees  didn't  win. 


DEWEY:  1  think  I  c  an  nee  ut  leant  part  of  your  point,  Ariel.  There  seems  to  be  a  part  of  what 
we  mean  by  'truth1  that  dependH  on  correspondence  with  the  way  things  are,  with  the  outside 
world.  That's  the  common  sense  view  of  things,  I  admit,  and  it's  often  useful.  But  let  me 
make  two  points.  First  of  all,  you  will  never  really  know  whether  the  Yankees  win  the 
pennant  except  by  comparing  your  prediction  with  what  other  people  say  about  who  won  the 
pennant.  You'll  always  be  comparing  your  statement  not  with  some  objective  reality,  but 
rather  with  other  people's  statements.  Just  as  you  would  in  the  case  of  the  chair. 

ARIEL:  True,  but  1  think  that  misses  the  point.  It  seems  to  me  that  you're  confusing  two 
separate  questions.  One  question  is  what  it  means  for  a  statement  to  be  "true."  To  put  it 
differently:  What  is  it  about  a  statement  that  makes  the  statement  true  or  false?  And  the 
answer  to  that  question,  I  submit,  is  that  a  statement  is  true  if  it  accurately  describes  what  it 
is  talking  about. 

DEWEY:  I  would  prefer  "faithfully."  True  statements  faithful  describe  or  interpret  to  what 
they're  talking  about.  They  do  justice— not  violence — to  the  subject  at  hand. 

ARIEL:  The  second  question  is  how  do  we  verify,  or  how  do  we  go  about  deciding,  whether  a 
statement  is  true.  We  can't,  as  you  say,  step  out  of  our  skins  and  lay  our  statement  alongside 
the  object  itself— and  I  very  much  doubt  whether  anyone  ever  thought  we  could.  So  we  use 
other  methods.  One  of  those  methods— although  certainly  not  the  only  method— is  to 
compare  our  beliefs  and  perceptions  with  other  people's  beliefs  and  perceptions.  That's  a 
method  for  determining  whether  a  statement  is  true.  And  I  agree  that  dialogical  coherence 
makes  good  sense  as  one  method  of  confirming  or  disconfirming  statements.  But  another 
legitimate  method  is  to  look  for  a  correspondence  between  the  thing  and  what's  said  about  it. 

DEWEY:  So  we're  headed  towards  a  dual— or  multiple— theory  of  truth  are  we?  Truth  can 
mean  different  things,  and  we  use  different  methods  to  show  it.  1  guess  I'm  not  strong 
enough  to  resist  you. 


IV 


MICHELE:  Now  that's  one  point  I've  got  to  agree  with.  But  it  seems  to  me  that  both  of  you 
have  absorbed  too  much  philosophy. 

DEWEY:  What  do  you  mean? 

MICHELE:  Just  notice  what's  happened.  The  question  we  started  off  with  was  whether  it's 
appropriate  to  judge  Linda  Esteban's  scholarship  by  "white  male"  standards.  Dewey  thought 
he  was  taking  the  open-minded,  liberal,  enlightened  view  by  arguing  that  we  shouldn't,  and 
that  the  standards  Harry  holds  so  dear  are  not  really  "objective"  or  "neutral."  If  he  had 
stopped  at  that,  things  would  have  been  fine.  But  instead,  he  had  to  go  ahead  and  offer  a  full- 
blown philosophical  analysis  of  what  was  wrong  with  Harry's  position,  and  then  to  suggest  a 
philosophical  alternative.  And  you  can  see  where  that  leads;  Dewey  just  set  himself  up  to  be 
taken  apart.  In  the  meanwhile,  we've  made  no  progress  toward  answering  the  initial 
question, 

DEWEY:  I  beg  your  pardon.  Mea  culpa.  Although  what  I  offered  was  hardly  a  "full-blown 
philosophical  analysis/1  Anyway,  what  should  I  have  done,  Michele? 
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MICHELE;  You  should  have  noticed  that  Harry's  insistence  on  applying  "merit"  criteria 
wasn't  a  Philosophical  position  at  all — as  he  himself  made  clear.  Harry  was  taking  a  political 
position.  And  the  proper  response,  therefore,  is  a  political  response.  Turning  the  issue  into  a 
matter  of  philosophy  just  diverts  attention  from  the  real  problem — and  dignifies  Harry's 
position  at  the  same  time. 

DEWEY:  I'm  still  not  sure  what  you  mean.  It  seems  to  me  that  Harry's  belief  in  a  particular 
set  of  "merit"  criteria  does  reflect  some  implicit  philosophical  assumptions — and  that  it's 
useful  to  bring  those  assumptions  out  into  the  open.  How  else  are  we  supposed  to  decide 
whether  Harry's  position  is  correct? 

MICHELE:  And  I  suggest  that  you've  gotten  trapped  in  a  philosophical  approach  to  the 
world.  And  the  result  is  that  however  good  your  intentions  may  be,  you'll  wind  up  endorsing 
something  like  Harry's  position,  even  if  you  do  it  unwittingly.  As  Ariel  has  just  shown.  And, 
in  fact,  as  the  statement  you  just  made  reflects.  After  all,  identifying  and  examining  nn 
author's  underlying  assumptions  is  precisely  the  kind  of  thing  that  Harry  thinks  good 
scholarship  ought  to  do.  Isn't  that  right,  Harry? 

HARRY:  It's  one  thing  good  scholarship  often  does,  yes.  And  I'm  glad  to  see  that  whichever 
route  we  take,  we  seem  to  end  up  back  at  something  like  my  position. 

MICHELE:  Oh,  no.  I  haven't  admitted  that.  There  is  another  alternative — quite  an  obvious 
onu,  really. 

IIAKKY:  Which  is? 

MICHELE:  To  appreciate,  as  I  said  a  moment  ago,  that  Harry's  position  is  really  political  in 
nature,  and  that  we  should  respond  accordingly. 

DEWEY:  Could  you  explain  that  a  little  more  clearly? 

MICHELE:  Well,  I  agree  with  Dewey,  of  course,  that  it  isn't  appropriate  to  judge  all 
scholarship  by  Harry's  so-called  "merit"  standards.  And  I  also  agree  that  Harry's  standards 
probably  reflect  (even  though  he  doesn't  realize  it)  a  "realist"  or  "objectivist"  understanding  of 
truth.  But  what  we  need  to  ask  about  Harry's  standards,  and  his  realist  understanding,  is 
not  whether  they  are  "true" — even  to  who  knows  what  exact  that  means! — but  rather  what 
political  consequences  follow  from  them. 

When  we  ask  that  question  systematically,  it  soon  becomes  apparent  what  the  actual 
significance  of  realism  is.  It's  a  device  for  justifying  structures  of  power  and  oppression. 
People  in  power  have  always  used  it  that  way.  The  medieval  church  that  tortured  and 
persecuted  heretics,  the  New  England  Puritans  who  executed  witches  and  Quakers,  the 
"enlightened"  Founders  who  defended  slavery  and  wrote  it  into  the  Constitution  and  who  left 
women  dependent  and  disenfranchised — the  ruling  powers  have  always  justified  their 
position  and  their  oppressive  practices  on  the  basis  of  a  realist  metaphysic.  What  they  were 
doing  was  always  in  accordance  with  some  "objective"  truth.  And  the  use  of  "objective" 
academic  standards  tod°v  to  denigrate  the  scholarship  of  women  and  scholars  of  color  is  just 
one  more  example  of  thin  venerable  practice.  It's  just  one  of  the  little  ironies  of  history  that 
the  most  articulate  defender  of  "realism"  in  this  room  today  happens  to  be  a  woman. 

ARIEL:  Alright,  then  let's  play  your  game  for  a  while.  We  won't  talk  philosophy,  just  politics 
and  power.  You  say  that  people  in  power  have  always  used  realism  to  justify  their  position; 
they've;  defended  their  practices  on  the  basis  of  so-called  "objective"  truths.  I  admit  that — 
cheerfully.  But  isn't  it  also  true  that  people  who  have  resisted  or  opposed  power  and 
oppression  have  also  done  so  on  the  basis  of  what  they  believed  to  be  "objective"  truths?  The 
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abolitionists  invoked — fiercely — what  they  believed  to  be  an  objective  higher  law.  When  they 
said  slavery  was  morally  abominable,  they  weren't  just  saying  that  they  didn't  have  a  taste 
for  it,  or  that  their  own  power  would  be  enhanced  by  the  elimination  of  slavery;  they  were 
saying  it  was  wronq.  The  same  was  true  of  those  who  worked  for  civil  rights  in  thiH  century. 
When  they  argued  that  segregation  was  unjust,  or  that  blackH  were  the  phyHical  and 
intellectual  and  moral  equals  of  whites,  they  weren't  just  making  a  power  grab.  Nor  were 
they  reporting  on  the  outcome  of  a  search  for  dialogical  coherence;  on  the  contrary,  whut  they 
said  often  was  not  compatible  or  coherent  with  views  and  beliefs  widely  accepted  in  American 
culture.  They  were  asserting  what  they  believed  to  be  true — in  a  realist  sense.  Am  I  wrong? 

DEWEY:  Michele,  is  it  OK  if  I  act  like  a  male  and  butt  in  on  this?  I  know  you  don't  like 
philosophy,  but  this  question  needs  a  philosophical  answer. 

MICHELE:  Go  ahead,  Dewey,  but  my  turn  next. 

DEWEY:  Ariel,  I'm  afraid  you're  confusing  ^  realist  position  with  a  person  being  committed 
to  the  belief  that  certain  things  are  true.  Realism  is  not  the  only  path  to  truth  or  to  a  person 
believing  they're  right.  Non-realists  aren't  sitting  out  there  with  no  principles,  no  beliefs, 
and  no  truth.  Believe  it  or  not,  there  are  postmodern  thinkers  out  there  who  are  social 
activists,  religious  and  political  reformers,  persons  totally  engaged  in  the  affairs  of  this 
world. 

ARIEL:  Still,  it  seems  to  be  true  that  both  those  in  power  and  those  who  are  resisting  power 
have  been  realists. 

MICHELE:  Historically  that's  been  true,  but  realism  has  been  the  dominant  worldview  of 
past  centuries.  I  think  that's  changing,  though,  with  more  dissident  and  activists  groups 
showing  a  variety  of  views  about  the  world.  But  that's  probably  beside  the  point.  The  point 
is  that  an  objectivist  realism  tends  to  favor  those  in  power  over  those  out  of  power* 

ARIEL:  Well  it  also  seems  true,  almost  by  definition,  that  people  with  power  will  more  often 
prevail  over  people  without  power.  But  I'm  not  sure  what  that  tells  us  about  realism.  In  fact, 
I'm  not  sure  whether  there's  much  of  a  connection  at  all  between  realism  and  struggle*  over 
power.  You  might  as  well  say,  "Human  beings  breathe  air  "  and  "The  world  in  unjust,"  Both 
statements  may  be  true,  but  there's  not  much  connection  between  them.  And  it  would  be  just 
plain  silly  to  link  the  statements  and  then  conclude  that  we  ought  to  take  a  stand  against 
breathing  air. 

MICHELE:  I'm  afraid  that's  a  rather  poor  analogy.  The  difference,  obviously,  is  that  there  is 
a  connection  between  realism,  or  a  belief  in  an  "objective  truth/'  and  power.  And  if  we  could 
dissolve  realinm,  if  we  could  expose  the  fallacy  of  "objective  truths,"  then  at  least  one  weapon 
that  the  powerful  typically  use  to  their  advantage  would  be  eliminated* 

ARIEL:  Yes,  I  suppose  you're  right  that  there  is  a  connection  between  realism  and  power.  I 
retract  my  analogy.  But  I  still  think  that  abandoning  realism  would  hurt  the  powerless  more 
than  the  powerful. 

MICHELE:  But  just  a  moment  ago  you  said  the  opposite.  You  agreed  that  realism  favors  the 
powerful  more  often  than  it  favors  the  powerless*  "Almost  by  definition/'  you  said. 

ARIEL:  Let  me  explain.  What  I  said  was  that  almost  by  definition  those  with  power  will 
usually  prevail  over  those  without  it.  That's  what  it  means  to  have  power.  But  it  doesn't 
follow  that  realism  is  what  allows  the  powerful  to  prevail.  Quite  the  opposite,  in  fact.  The 
powerful  don't  need  truth.  After  all,  they  already  have  power:  they  have  clubs,  or  bullets,  or 
money.  It's  the  powerless  who  need  truth  because,  f;o  use  your  metaphor,  it's  the  only  weapon 

CJ5 


they  have. 


MICHELE:  That'n  wonderful  idealistic  rhetoric.  Unfortunately,  it  doesn't  work  very  well  in 
the  real  world.  Fundamentalists,  for  example,  are  surely  realists,  yet  they  tend  to  be  among 
the  most  militant  of  peoples.  Realism  and  power  tend  to  go  hand  in  hand.  You  say  the 
powerful  don't  need  truth  since  they  have  power.  But  their  biggest  fear  is  that  t'.iey  might 
lose  power,  so  they  take  great  pains  to  control  how  the  story  is  told.  And  since  they  believe  in 
only  One  True  Story,  they  have  a  vested  interest  in  realism  as  a  vehicle  for  preserving  the 
status  quo. 

The  logic  goes  something  like  this: 

— There  is  only  One  True  Way  of  seeing  things. 

— The  Ruling  Society  tells  things  that  way — the  way  things  are. 

— If  you  don't  conform  to  the  Ruling  Society's  story,  then  there  is  something  wrong  with  you. 

DEWEY:  So  here  we  are  back  to  a  healthy  pluralism.  Out  of  our  little  dialogue  some  truths 
seem  to  have  emerged.  Ariel  has  persuaded  me  to  admit  a  role  for  correspondence  in 
thinking  about  truth,  and  Michele  has  shown  the  pragmatic  and  political  dangers  of  in 
objectiviflt  view  that  discourages  multiple  perspectives.  I'm  not  sure  if  I'm  postmodern,  hut  1 
can  definitely  see  some  value  in  pluralism. 

THE  DEAN:  I'm  sorry  to  have  to  interrupt  at  this  point.  This  has  been  an  illuminating 
discussion,  I'm  sure.  However, !  mentioned  that  some  people  might  have  to  leave,  and  in  fart 
1  have  a  meeting  with  the  Victv Chancellor  in  about  five  minutes,  so  I'm  afraid  we're  goi'ig  to 
have  to  adjourn  without  reaching  a  decision  on  the  matter  at  hand.  And  I  don't  think  we 
should  wait  too  ?ong  on  this.  Can  I  just  ask:  ifov  soca  do  you  all  think  you  may  be  ready  to 
make  a  decision  on  this  matter? 

MICHELE:  I'm  ready  to  vote  right  now. 

HARRY:  I'm  not.  But  I  do  have  a  request.  If  Dewey  and  Arie!  would  mal>e  up  a  reading  list 
naming  what  they  each  regard  as  the  ten  most  important  books  cn  the  nature  of  "is  uth,"  Til 
study  those  just  as  soon  as  I  find  the  time.  Then  I'll  be  prepared  to  vote. 

ARIEL:  And  by  then  the  rest  of  us  will  have  passed  into  the  realm  of  higher  truth. 

MICHELE:  You  see  where  philosophy  leaves  you. 

THE  DEAN:  Seriously,  could  we  meet  tomorrow  at  noon  for  about  an  hour?  I'll  provide  lunch. 
That  will  give  Harry  almost  24  hours  to  figure  out  the  nature  of  truth. 

DEWEY:  That  should  be  plenty  of  time  for  him.  Even  though  he'll  he  ye  to  interrupt  his 
ruminations  for  a  half-hour  to  catch  "The  Siirrwons* 

HARRY:  It  won't  be  an  interruption.  That's  probably  where  Vll  find  the  answer. 

ARIEL:  If  it  comes  to  that,  then  on  this  issue — and  only  on  this  issue — I'd  advise  you  to  trust 
Bart's  judgment.  Not  Lisa's. 

HARRY:  Don't  worry.  I'd  have  done  that  anyway. 

MICHELE:  We  know. 


THE  DEAN:  I'll  nee  you  all  tomorrow.  At  noon. 

i  j  e 


(Meeting  concludes,) 
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The  propositions  of  those  models  have  raised  important  questions  among  educators  generally,  and 
instructional  designers  specifically.  For  example.  "  What  is  the  best  instructional  system  that  should  be 
used  for  inducing  learners1  cognitive  strategies?  is  it  a  system  in  which  learners  should  be  directed  to 
generate  their  own  cognitive  strategy  activators  (GS).  or  a  system  in  which  teachers  ( researchers,  or 
designers)  should  prepare  those  activators  and  give  them  to  learners  (ES)?  What  is  the  best  position  of 
cognitive  strategy  activators  should  be  inserted  in,  is  it  before  instruction,  during,  or  after  instruction?  What 
is  the  best  mode  of  cognitive  strategy  activators  that  should  be  used,  is  it  a  visual  (concrete)  mode,  or  verbal, 
written  (abstract)  mode? 

To  address  the  above  questions,  researchers  have  conducted  numerous  experimental  studies  to 
investigate  the  effectiveness  of  cognitive  strategy  activators  In  terms  of  their  instructional  systems  delivery, 
positions,  and  modes.  In  those  studies,  the  cognitive  strategy  referes  to  intellectual  functioning  of  the 
human  mind  and  the  abilities  to  use  one's  knowledge  through  such  activities  as  remembering, 
comprehension,  focussing  attention,  and  processing  information  ( Babbs,  &  Moe,  1983);  whereas  the 
cognitive  strategy  activators  have  been  used  to  mean  as  instructional  strategy  components  which  either 
request  or  force  students  to  use  a  cognitive  strategy  such  as  adjunct  questions,  organizers,  underlining 
etc...  (  Reigeiuth  &  Darwazeh.  1982 ). 

With  respect  to  the  cognitive  strategy  activators1  system,  resrearchers  have  investigated  two  kinds  of 
system: 

1  •  The  first  system  was  known  as  an  Embedded  Cognitive  Strategy  System  (ES)  in  which  learners  are 
forced  to  use  a  given  cognitive  strategy  activator  that  was  prepared  by  the  teacher  (instructor,  designer,  or 
researcher).  For  example,  requesting  learners  to  answer  questions  that  are  given  to  them  by  the  instructor 
to  help  them  to  acquire,  retain,  and  retrieve  different  kinds  of  knowledge  is  an  embedded  cognitive  strategy 
procedure. 

2-  The  second  system  was  known  as  a  Generative  Cognitive  Strategy  System  (GS)  in  which  learners  are 
directed  to  generate  a  certain  cognitive  strategy  activator  by  themselves.  For  example,  requesting  learners 
to  generate  questions  to  help  them  to  aquire,  retain,  and  retrieve  different  kinds  of  knowledge  is  a 
generative  cognitive  strategy  procedure. 

With  respect  to  the  position  of  cognitive  strategy  activators,  researchers  have  investigated  mainely  two 
positions  of  cognitive  activators: 

1*  Before  (or  Pre)  instruction  in  which  the  cognitive  strategy  activators  are  presented  before  beginning 
instruction. 

2-  During  and  After  (or  Post)  instruction  in  which  the  cognitive  strategy  activators  are  presented  during 
instruction  or  after  instruction. 

With  respect  to  the  mode  of  cognitive  strategy  activators,  researchers  have  also  investigated  two  kinds 
of  modes: 

1  -  Visual  (or  Concrete )  cognitive  strategy  activators  in  which  information  of  it  is  presented  in  figures,  maps, 
flow  chart,  diagrams,  trees,  etc.. 

2-  Written  (or  Abstract )  cognitive  strategy  activators  in  which  information  of  it  is  presented  in  a  narrated 
language. 

The  results  of  previous  studies  concerning  either  system  of  cognitive  strategy  activators,  positions, 
and  modes  have  been  inconsistent.  This  inconsistency  was  due  to  the  effect  of  different  instructional 
variables  (or  conditions)  that  existed  during  experimentation.  These  variables  were  centered  around  three 
major  elements. 

1-  Learner  characteristics  ( i.e.,  high  vs.  medium  vs.  low  ability  learners ;  field-dependent  vs.  field- 
independent  learners). 

2-  Content  characteristics  ( I.e.,  organized,  vs.  random;  familiar  vs.  unfamiliar;  content  sequenced  from 


specific  to  general  vs.  from  general  to  specific  information). 

3-  Levels  of  learning  ( i.e.,  high  vs.  medium  vs.  low  level  of  learning). 

Research  Problem: 

Oesptte  the  fact  mat  there  were  a  tremendous  number  of  studies  that  have  been  conducted  on 
cognitive  strategy  activators,  there  was  no  single  research  that  synthesizes  the  results  of  those  studies  to 
see  whether  there  is  a  common  trend  of  those  results  that  helps  us,  as  Instructional  designers,  to  design 
msturction  based  on  them. 

On  the  other  hand,  the  cognitive  strategy  activators  are  still  not  considered  in  the  models  of 
instructional  design  (e.g.,  Dick  &  Carey,  1990;  Gagne,  Briggs,  &  Wager,  1992),  except  for  an  attempt  by 
West  and  his  colleagues  ( West,  Fanner,  and  Wolff,  1992)  to  put  a  template  of  instructional  design  based  on 
cognitive  science  principles.   Their  attempt  was  a  good  trial,  but  still  did  not  specify  a  clear  and 
comprehensive  prescriptive  model  for  designing  instruction  with  respect  to  cognitve  activators.  Thus,  the 
gener aLaim  of  the  current  research  is  to  try  to  create  two  prescriptive  instructional  design  models  based  on 
the  general  trend  of  the  results  of  those  previous  studies  which  were  conducted  on  cognitive  strategy 
activators  between  1960's-1990's. 

Research  objectives: 

The  objectives  of  the  current  research  are  the  foitowing: 

1  -  Specifying  conditions  for  the  best  use  of  cognitive  strategy  activators  in  terms  of  their  systems  (ECS  vs. 
GCS) .  positions  (Pre  vs.  Post),  and  modes  (Visual  vs.  or  Written ),  with  their  relation  to  three  instuctionai 
variables:  (a)  Learner  characteristics,  (b)  Content  characteristics,  and  (c)  Levels  of  learning. 

2-  Proposing  two  prescriptive  instructional  design  models  of  cognitive  strategy  activators  to  be  used  as  a 
guideline  for  either  instructional  designers,  developers,  or  teachers.  The  model  could  also  be  Used  as  a 
guideline  for  learners  on  how  to  develop  effective  learning  strategies  during  studying. 

3-  Giving  recommendations  for  future  research. 

Significance  of  the  research: 

The  current  research  represents  a  small  but  hopefully  important  cummulative  contribution  to  our 
understanding  of  how  cognitive  strategy  activators  can  best  be  used  to  improve  students  learning.  The 
suggested  modefeof  this  research  may  be  of  immediate  use  to  instructional  designers  and  classroom 
teachers  who,  in  addition  to  making  the  best  use  of  existing  prose  materials  and  software  programmes,  are 
also  concerned  with  fostering  effective  independent  study  skills  in  students.  Instructional  developers, 
including  those  at  ail  levels  and  in  ail  contexts  of  education  and  training,  stand  to  benefit  through  an 
increased  understanding  of  the  role  of  cognitive  activators  in  enhancing  learning  and  instruction. 

Research  question*: 

The  current  research  is  trying  to  answer  the  following  three  broad  questions: 
Question  1.  Under  what  conditions  are  Embedded  versus  Generative  cognitive  strategy  activators 
effective? 

Question  2  Under  what  conditions  are  Pre  versus  Post  cognitive  strategy  activators  effective? 
Question  3.  Under  what  conditions  are  visual  versus  written  cognitive  strategy  activators  effective? 
Question  4.  How  can  we  design  instruction  with  a  consideration  of  cognitive  strategy  activators? 
Research  hypothsee: 

Based  on  the  reveiw  of  the  literature  on  cognitive  strategy  activators,  the  author  proposed  the  following 
hypotheses.  These  are: 
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Hypothesis  1 . 

Having  learners  generate  or  use  cognitive  strategy  activators  would  lead  them  to  a  better 
achievement,  hence,  to  a  better  understanding  and  learning  ( Peper  &  Mayer,  1986;  Rothkopf, 
1966, 1970;  Rigney.  1978;  1980;  Wittrock,  1974a,  1974b). 

Hypothesis  2. 

The  preference  for  using  a  certain  system  of  cognitive  strategy  activator  { Embedded  or 
Generative), using  a  certain  position  (  Pre  or  Post ),  or  using  a  certatin  mode  (visual  or  written ) 
depends  on  its  interaction  with  different  the  instructions!  variables  that  may  exist  in  the 
instructuonal  situation,  such  as:  learner's  characteristics,  contents  characteristics,  and  levels  of 
learning. 

Rationale  for  H1  : 

1-  Having  learners  generate  or  use  cognitive  strategy  activators  will  lead  them  to  pay  more  attention  to 
the  presented  materials  to  be  learned,  so  the  learner  can  process  the  information  more  deeply  and 
store  it  in  long  term  memory  ( Barry,  1974;  Frase,  1970;  Norman  &  Lindsay,  1977;  Osborne  &  Wittrock, 
1983:  Peper  &  Mayer.  1986). 

2-  Having  learners  generate  or  use  cognitive  strategy  activators  will  lead  them  to  get  involved  in 
additional  cognitive  processes,  and  to  actively  relate  the  materials  to  be  learned  to  their  existing 
knowledge  (AusubeU960, 1968;  Mayer,  1979;  Peper  &  Mayer,  1980;  Shreger  &  Mayer,  1989; 
Wittrock.l974a.1974b.  1990). 

3-  Having  learners  generate  or  use  cognitive  strategy  activators  will  increase  their  levels  of  motivation 
( Attendance,  Reievence,  Confidence.  &  Satisfaction ),  hence,  to  acquire,  retain,  and  retrieve  different 
kinds  of  knowledge  on  different  levels  of  learning  ( Keller,  1983;  Reigeiuth,  1983;  Wittrock.1990). 

Rationale  for  H2: 

Since  teachers  (or  instructions!  designers)  are  using  different  kinds  of  instructional  systems,  different 
positions,  and  different  modes  of  cognitive  strategy  activators,  and  dealing  at  the  same  time  with  diffemet 
learners  characteristics,  different  content  characterestics,  and  different  levels  of  learning;  it  is  logical  to  get 
different  effects  of  cognitive  strategy  activators  on  learining.  What  we  intended  to  find  in  this  research  is 
whether  there  is  a  general  trend  of  those  previous  studies'  results  with  respect  to  those  Instructional 
variable*  The  aim  of  doing  such  job  Is  to  create  general  prescriptive  design  models.  Hopefully,  these 
findings  can  also  stimulate  other  researchers  to  conduct  more  studies,  hence,  to  derive  more  principles  and 
add  them  to  the  suggested  models  for  the  better  use  of  cognitive  strategy  activators. 


BMeatCb  Methodology 

While  the  author  has  been  at  different  Un'versities  in  the  USA  as  a  doctoral  student,  then  as  a  visiting 
professor,  she  sleuthed  Information  and  collected  data  about  cognitive  strategy  activators  which  is  within 
her  research  Interest.  The  Information  was  collected  from  different  resources  at  different  times.  These 
resources  were  basically  the  following: 

1-  The  Bird  Library  at  Syracuse  University. 

2-  The  Randall  Library  at  the  University  of  North  Carolina  at  Wilmington. 

3-  The  Main  Library  at  the  University  of  Georgia  at  Athens. 

4-  ERIC  Documentation  at  SU.  UNCW.  and  UGA. 

5-  Papers  presented  at  different  Annual  meetings  tor  education  held  at  USA  at  different  times. 

6-  Conducting  few  studies  on  cognitive  strategy  activators  by  the  author  herself  (See  the 
reference  section). 

From  those  resources,  the  author  was  able  to  find  approximately  one  hundred  and  fifty  experimental 
studies  and  theoretical  research  on  cognitive  strategy  activators.  They  were  found  in  different  books, 
periodicals,  dissertation  abstract  (International,  mlcroflchf.  wvfcing  papers,  papers  presented  at  different 

&/1 


annual  educational  meetings  that  were  held  in  the  United  States  of  America.  Thus,  the  author  considers 
those  studies  as  the  ad-hoc  samolj  of  this  research.  The  methodolgy  she  followed  in  this  research  was  the 
descriptive  methodology  of  research. 

The  procedure  that  the  author  used  in  conducting  this  research  consisted  of  the  following  steps: 

1-  Reviewing  a  number  of  previous  theoretical  and  experimental  studies  that  were  avialable  to  the  author, 
which  were  conducted  between  1960s  -  1990s  on  cognitive  strategy  activators  in  terms  of  their  instructional 
systems,  positions,  and  modes. 

2-  Classifying  those  previous  experimental  studies  into  three  major  categories: 

a-  The  studies  that  have  manipulated  the  instructional  system  of  cognitive  strategy  activators: 
Embedded  versus  Generative  Cognitive  Strategy  Activators'  Systems  (ECS  vs.  GCS). 

b-  The  studies  that  have  manipulated  the  position  of  cognitive  strategy  activators:  Pre-  versus  Post- 
position). 

c-  The  studies  that  have  manipulated  the  mode  of  cognitive  strategy  activators:  visual  (concrete) 
versus  written  (abstract)  modes. 

Therefore,  any  expenmentai  study  or  theoretical  research  that  did  not  fit  in  these  categories  was 
excluded,  regardless  its  relation  to  cognitive  strategy  activators. 

3-  Summarizing  the  results  of  those  studies  according  to:  the  author* same,  published  yeat  the  aim  of  the 
study,  the  sample  size,  and  the  indepependent  and  dgp^Ant  vanahi^  and  then  the  cesutts. 

4-  Synthesizing  (or  categorizing)  the  results  of  the  experimental  studies  according  to  cognitive  strategy 
activators  systems  (ECS  vs.  GCS),  positions  (Pre  vs.  Post  Instruction),  and  modes  (visual  vs.  written). 

5-  Drawing  a  conclusion  from  each  cognitive  strategy  activator's  studies.  Then,  a  general  conclusion  of  all 
studies1  results  was  drawn  at  the  end  of  each  category.  The  general  conclusions  represented  the  general 
trend  of  the  results  of  the  previous  studies.  Consequently,  two  prescriptive  instructional  design  models 
were  proposed  on  oognrtive  strategy  activators  to  be  as  a  guide  for  Instructional  designers,  developers, 
teachers,  and  learners,  in  their  designing  instruction,  teaching,  or  studying. 

The  author  found  from  the  review  of  the  literature  that  the  most  frequent  cognitive  strategy  activators 
which  have  been  examined  by  researchers  were  alphapeticaliy  the  following :  Adjunct  questions, 
Advanced  organizers,  Analogies,  Diagrams,  Heading.  Mental  Images,  Information  mapping,  Meaning 
generation,  Note-Taking,  Outlines.  Pictures,  Stories,  Summaries,  Synthesis,  Titles  and  Sentences.  Thus, 
they  were  chosen  to  be  the  core  of  the  current  research. 

For  AECT  presentation  within  a  limited  time,  we  will  be  satisfied  to  present  just  the  name  of  the  authors 
who  conducted  those  studies  and  me  year  of  publication,  plus  the  general  trend  of  their  results.  A  full 
report  of  this  research  containing  the  detailed  reviews  of  each  of  the  previous  studies  will  be  submitted  to 
the  'Review  of  Educational  Research"  Journal. 

Literature  Review  and  Conclusions 

1-  Embedded  vereue  Generative  Cognitive  Strategy  Activator  Studies: 

Much  of  pervious  studies  have  manipulated  the  usage  of  cognitive  strategy  activators  as  (ECS)  only. 
Several  studies  have  manipulated  these  activators  as  (GCS)  only.  But  a  limited  number  of  studies  have 
made  a  direct  comparison  between  ( ECS)  versus  ( GCS), 

In  the  current  research,  we  are  concerned  mainly  with  ttm+Jrti  category  of  studies  which  compared 
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(ECS)  with  ( GCS)  by  using  various  kinds  of  cognitive  strategy  activators.  The  review  of  previous  studies 
coverd  the  following  cognitive  strategy  activators  alphapetically:  ' 

-  Adjunct  Questions  ( e.g..  Andre  &  Anderson.  1978-1 979;  Crouse  &  ldstein.1972;  Darwazeh.1982;  Oavey 
&  McBnde.1986;  Dreher  &  Gambrell.  1982;  Duell.1977;  Fraze  &  Schwartz.  1975;  Hetfeldt  &  Laiik.  1976; 
King.  1991;  Manzo.  1970;  Owens.  1977;  Seretny*  Dean,  1986). 

-  Advance  Organizers  ( e.g..  Ausubel, 1960;  AusubeU964;  Ausubel  &  Fitzgerald.  1961 ,1962;  Ausubel  & 
Youssef.  1963;  Darwazeh,  1988;  Oinnel  &  Glover.1985;  OiVesta  &  Peverfey.  1984;  Doctorow,  Wittrock.  & 
Marks,  1978;  Jonassen  &  Cole.  1993;  Jonassen,  Cole,  &  Bamtord,l992;Kenny,  1992;  Kbzlow  &  White, 
1980;  Luiten,  1980;  Mayer,  1978.1979;  Schmid  &  Telaro.  1990;  Snapp  &  Glover.  1990;  Wittrock,  1974a. 
1974b;  Wittrock  &  Carter.  1975). 

-  Heading  and  Sentences  ( e.g..  Dee  Lucas  &  Di Vesta,  1980;  Jonassen.  et  al„  1985). 

-  Images  vs.  Pictures  ( e.g..  Anderson  &  Kulgavey.1972;  Bull  &  Wrttrock.1973;  Carrier,  et  a!..  1983a.  1983b; 
Essen  &  HamakeM99Q;  Pressiey,  1976 ). 

-  information  Maps  (  e.g..  Ally  &  SZabo.  1992;  Dansereau,  et  at.,  1979;  Mayer,  et  al.,1984;  McCagg  & 
Dansereau.  1991;  Lenz,  1992;  Ragan,  1991). 

-  Note-Taking  ( e.g..  Bretzing.  et  at.,  1987;  Darwazeh.1993;  Kiowra  &  Prank.1988;  Paper  &  Mayer.  1986; 
Simbo,  1988;  Shrager  &  Mayer,  1989). 

«  Outlines  ( e.g..  Tuckrnan,  1993). 

•  Summaries  ( e.g..  Annis,  1985;  Harris.  1992;  Hooper ,  dt  al.  1992;  Spurlin,  et  ai.,  1988;  Wittrock  & 
Alesandnni,  1990). 

*  Underlining  ( e.g..  Blanchard  &  Mikkelson.  1987;  Idestein  &  Jenkins.  1972;  Rlckards  &  August.  1975 ). 
General  Conclusions: 

After  reviewing  the  above  cited  studies  on  cognitive  strategy  activators,  we  found  that  the  generative 
system  is  generally  more  effective  in  increasing  students'  learning  than  the  embedded  system.  At  ,the  same 
time,  each  system  has  privileges  over  the  other  under  certain  conditions.  In  other  words,  the  effectiveness 
of  embedded  versus  generative  cognitive  system  or  vice  versa  interacts  with  different  instructional 
variaibles  such  as,  learner  characteristics,  content  characteristics,  and  levels  of  learning. 

in  short,  we  can  state  according  to  the  above  studies  that  the  Embedded  Congnitive  Strategy 
Activators  are  effective  under  these  conditions; 

1-  Low  and  medium  ability  students  ( Ausubel  &  Fitzgerald, 1 961 1 1962;  Lenze,  1992;  Mayer,  1978, 1979; 
Schmid  &  Telaro,  1990;  Seretny  &  Dean,  1986;)* 

2-  Reld-dependent  students  ( Carrier,  et  al.  1983;  Kiewra  &  Frank,  1988;). 

3-  Un-trained  students  ( Kiewra  &  Frank.  1988;  Simbo.  1988 ). 

4-  Organized  learning  content  ( Di\Ar  'ia  &  Perveriy,  1984;  Wittrock  &  Carter,  1975). 

5-  Familiar  learning  content  (  Peper  &  Mayer,  1986;  Sharger  &  Mayer,  1989). 

6*  Low  and  medium  levels  of  learning  (Ally  &  Szabo,  1992;  Davey  &  McSride,  1986;  Darwazeh,  1982; 
Darwazeh,  1988;  Jonassen.  et  ai.  1992;  Jonassen  &  Cole.  1993;  Jonassen,  et  al.  1985;  Kiewra  &  Frank, 
1988;  Larue.  1992;  Manzo,  1970;  Peper  &  Mayer,  1986;  Pagan,  1991 ). 

Whereas  the  Generative  Cognitive  Strategy  Activators  are  effective  under  these  conditions; 

1-  High  ability  students  ( Annis,  1985;  Blanchard  &  Mikkelson,  1987;  Bretzing.  et  al.  1987;  Doctorow.  et  ai. 
1978;  Len2e.  1992;  Wittrock  &  Alesandnni.  1990). 

2-  RekHndependent  students  ( Carrier,  et  al.  1983;  Kiewra  &  Frank,  1988). 
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3-  Trained  students  (  Andre  &  Anderson,  1978-1979;  Essen  &  Hamaker,  1990;  King.1991;  Mayer,  et  al. 
1984;  McCagg  &  Dansereau.1991;  Pressley.  1976;  Simbo,  1988;  Tuckman,  1993). 

4-  Random  learning  content  (  DIVesta  &  Peverley,  1984;  Wittrock  &  Carter,  1975). 

5-  Unfamiliar  learning  content  ( Peper  &  Mayer.  1986;  Shrager  &  Mayer,  1989). 

6-  Low.  medium,  &  high  levels  of  learning  (Blanchard  &  Mikkelson,  1987;  Bretzing,  et  al.  1987;  Bull  & 
Wittrock,  1973;  Carrier,  et  al.  1983;  Davey  &  McBride.1986;  Dee-Lucas  &  DiVesta.1980;  OiVesta  & 
Perveriey.  1984;  Oinnel  &  Glovar,  1985;  Doctorow,  et  al.  1978;  Dueil,  1977;  Essen  &  Hamaker,  1990; 
Fraze  &  Schwartz,  1975;  Helfefdt  &  Lalik.1976;  Hooper  &  Rysavy,  1992;  Idestein  &  August,  1975; 
Jonassen,  et  al.  1985;  King,  1991 ;  Kulhavey,  1972;  Manzo,  1970;  Mayer,  et  al.  1984;  Peper  &  Mayer, 
1986;  Pressley,  1976;  Richards  &  August.  1975;  Snapp  &  Glover,  1990;  Simbo.1988;  Shrager  &  Mayer. 
1989;  Spurlin,  et  al.  l988;Tuckman.1993;  Wittrock  &  Carter,  1975;  Wittrock  &  Alesandrini,  1990)  (See^ 
Figure  1). 

 Insert  Figure  1  about  here  

2-  Position  of  cognitive  strategy  activator  Studies: 

A  good  number  ot  studies  have  investigated  the  effect  of  the  position  ot  cognitive  strategy  activators 
on  learning  especially  by  using  adjunct  questions  or  advance  organizers.  The  aim  of  those  studies  was  to 
see  whether  pre  position  is  more  effective  than  post  position  or  vice  versa,  and  when.  The  review  of 
previous  studies  coverd  the  following  cognitive  strategy  activators  alphapeticaliy: 

-  Adjunct  questions  (  e.g..  Ander,  et  al.,  1980;  Anderson  &  Biddie,  1975;  Darwazeh,  1982;  Feiker& 
Dapra,  1 978;  Frase.  1967.1968;  Hedgins.  et  al.,1979;  Memory,  1980;  Rickards,l976;  Rickards  4  Hatcher. 
1978;  Sanders,  1973;  Shavelson.  et  al.,  1974). 

•  Organizers  ( e.g.,  Alexander,  et  al.,1979;  Anderson. 1973;  Bauman,  Glass.  &  Hamngtom.1969; 
Darwazeh.  1988;  Graber,  Means.  &  Johnsten.1972;  Mayer.1978;  Simmons,  et  al.  1988;  Woodward,  1966). 

•  Synthesizer  ( e.g.,  Careson  &  Reigeluth.  1983;  Frey  &  Reigeluth.  1981). 

•  Pictures  ( e.g.,  Carrier  et  ai.,  1983). 
General  Conclusions: 

The  major  conclusion  of  the  previous  studies  on  pre  and  post  cognitive  strategy  activators  was  that  the 
post  cognitive  strategy  activators  are  more  effective  in  increasing  teaming  than  the  pre  ones.  However, 
each  position  has  prtvi  Itges  over  the  other  under  certain  conditions.  In  other  words  the  effectiveness  of 
pre  versus  post  cognitive  strategy  activators  or  vice  versa  interacts  with  different  instructional  variables.  In 
short,  we  can  state  according  to  the  above  cited  studies  that  the  pre  CSA 's  are  effective  under  these 
conditions: 

1-  Low  ability  students  (  Darwazeh,  1982;  Memory,  1983;  Sanders.  1973:) . 

2-  with  teaming  content  sequenced  from  specific  to  general  information  (  Frey  &  Reigeluth,  1981). 

3-  Low  levels  of  learning  (  Anderson  &  Biddie.  1975;  Andre,  et  al.  1980;  Darwazeh  &  Riegeluth.  1982; 
Fraze.1967.1968;  Rothkopf.1966;  Rothkopf  &  Bisbicos.  1967;  Sanders.  1973;  Simmons,  et  ai.  1988). 

Whereas  the  post  CSA's  are  effective  under  these  conditions: 

1-  Medium  and  High  ability  students  ( Darwazeh,  1982;  Hudgins,  et  ai.  1979;  Richards  &  Hatcher,  1978; 
Sanders,  1973). 

2-  With  learning  content  sequenced  from  general  to  specific  information  ( Careson  &  Riegeluth.  1983;  Frey 
&Riegeluth,  1981). 

3-  Low.  medium,  arWhigh  levels  of  teaming  (  Anderson  &  BkkJle.1975;  Andre,  et  al.  1980;  Bauman.  et  al. 
1969:  Careson  &  Riegeluth.  1983;  Darwazeh  &  Riegeluth.  1982;  Darwazeh,  l988:Fraze,  1967, 1968; 
Rothkopf.  1966;  Rothkopf  &  Bisbicos,  1967;  Sanders.  1973;  Sagaria  &  DiVesta.  1978  )  (See  Figure  2). 

 Insert  Figure  2  about  here  
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3-  Mod*  of  cognitive  strategy  activator  tiudfee: 

The  studies  on  the  mode  ot  cognitive  strategy  activators  have  manipulated  mainly  the  visual  (concrete), 
versus  written  (abstract)  cognitive  strategy  activators.  The  review  ot  previous  studies  coverd  the  following 
cognitive  strategy  activators  alphapetically: 

-  Advance  Organizers  ( e.g.,  Barnes  &  Clawson.1975;  Barron.1971 ;  Corkittt  et  ai.,1988;  Kenny.  1992; 
Lucas.1972;  Weisberg,  1970). 

-  Synthesizers  (  e.g.,  McLean,  et  al.  1983;  Chao  &  Reigeiuth.  1986;  Rigney  &  Lutz.  1976;  Holmes.  1987; 
Beentjes  4  Van  der  Voot.  1992). 

-  Analogies  ( e.g..  Dean,  et  al.,1980;  Newby  &  Steprch.1991). 

•  Maps  &  Diagrams  ( e.g..  Cha  &  Dwyer.1991 ;  Lambiotte  &  Dansereau.  1992;  Lambiotte.  199;  Winn  & 
Sutherland.  1989;  Winn,  et  al..  1991}. 

General  Conclut  Ions: 

According  to  the  review  ot  the  above  studies,  we  found  that  using  visualized  cognitive  strategy 
activators  are  more  effective  than  using  verbal  or  written  cognitive  strategy  activators.  On  the  other  hand, 
varying  the  mode  ot  cognitive  strategy  activators  depends  on  different  instructional  variables  that  might 
interact  with  the  mode.  Therefore,  the  verbal  cognitive  activators  are  effective  under  certain  conditions 
which  are  different  from  the  conditions  that  the  written  cognitive  activators  are  effective  under. 

Generally,  we  concluded  according  to  the  above  results  that  the  visual  cognitive  strategy  activators  are 
effective  under  the  following  conditions: 

1-  Low  ability  students  (  Holms.  1987;  Winn  &  Sutherland.  1989;  Winn,  et  al.  1991). 

2-  With  unfamiliar  learning  content  ( Lambiotte  &  Dansereau.  1992;  Lambiotte,  1993). 

3-  Low  and  medium  levels  of  learning  ( Barnes  &  Clawson,  1975;  Beenties  &  Van  Voot.  1992;  Chao  & 
Riegeluth.1986;  Corkill,  et  al.  1988;  Dean,  et  al.  1990;  Holms.  1987;  Kenny,  1992;  McLean.  etaJ.  1983; 
Newby  &  Stepich,  1991 ;  Rigney  &  Lutz.  1976;  Winn  &  Sutherland.  1989). 

Whereas  the  written  cognitive  strategy  activators  are  effective  under  the  following  conditions: 

1-  High  ability  students  ( Holms.  1987). 

2-  With  familiar  learning  content  ( Lambiotte  &  Dansereau.  1992;  Lambiotte.  1993) . 

3-  High  levels  of  learning  (Beenties  &  Van  Voot.  1992  (See  Figure  3). 

 Insert  Figure  3  about  here  

Dlacuaalon 

Before  discussing  the  findings  of  this  research.  It  is  worthwhile  to  bear  in  mind  three  important  points:  1) 
the  current  research  was  based  prtmarty  on  a  sample  but  not  on  all  studies  that  have  been  conducted  on 
cognitive  strategy  activators,  2)  the  findings  represent  the  general  trend  of  previous  studies1  results  which 
were  found  In  most  of  the  reviewed  studies  but  not  In  every  study,  and  3)  the  findings  were  organized  with 
respect  to  three  major  instructional  variables:  a-  learner  characteristics,  b-  content  characteristics,  and.  c- 
levels  of  learning.  This  organization  does  not  mean  that  there  were  no  other  variables  that  might  Interact 
with  cognitive  strategy  activators.  These  variables  are  used  just  because  they  were  most  frequently 
examined  by  previous  researchers. 

Thus,  researchers  are  recommended  to  take  into  consideration  some  other  Instructional  variables  In 
future  research  such  as.  learner's  level  of  anxiety  (high,  medium,  or  low),  learner's  level  of  motivation  ( high, 
medium,  or  low),  type  of  personality  (introvert,  or  extrovert),  type  of  instructional  content  ( facts,  principles, 
concepts,  or  procedures),  type  of  delivery  system  ( text  bock,  computer,  interactive  video  disc.  TV)  etc... 


1-  Ebmedded  va.  Generative  Cognitive  Strategy  Activators'  System: 

Ths  review  of  previous  studios  on  embedded  vs.  generative  cognitive  strategy  activators  revealed  six 
important  results  with  relation  to:  1 )  learner  charactenstics,  2)  content  characteristics,  and  3)  levels  of 
learning. 

1 )  With  respect  to  the  learner  characteristics,  the  results  of  this  research  have  indicated  that  the  Embedded 
System  (ECS)  was  effective  with  low  and  medium  ability  students;  Field-dependent  students,  and  un- 
trained students;  whereas  the  Generative  System  (GCS)  was  effective  with  high  ability  students,  Field- 
independent  students,  and  trained  students. 

One  explanation  of  these  findings  could  be  that,  students  under  ECS  are  required  to  manipulate  the 
cognitive  activators  that  have  been  prepared  by  teachers.  Hence,  students  in  this  system  are  dependent 
on  teachers  and  expect  to  receive  such  activators  from  them,  especially  when  they  don't  own  the  required 
ability  to  do  this  task  by  themselves  ( e.g.,  Oarwazeh.  1982;  Carrier,  et  al.  1983, 1984;  Orane,  et  ai.  1989 ). 
On  me  othe.-  hand,  once  they  receive  these  activators  from  a  teacher  they  don't  need  to  be  trained  in  how 
to  generate  these  activators;  because,  the  teacher  is  expected  to  give  them  ideal  ones  due  to  their 
competence  and  mature  experience  in  teaching. 

But  the  opposite  was  true  with  students  under  (GCS)  in  which  students  are  required  to  generate  the 
cognitive  activators  by  themselves.  Thus,  students  in  this  system  are  independent  of  the  teachers  and  are 
expected  to  to  do  this  job  by  themselves.  And  because  it  is  not  an  easy  job  to  do,  students  need  to  have  a 
high  learning  ability  or  to  be  trained  to  do  this  job.  In  other  words,  it  is  not  easy  for  everybody  to  generate 
cognitive  activators  unless  they  are  capable  enough  to  do  it.  and  have  an  independent  cognitive  style.  This 
explanation  was  supported  by  Wrttrock  (Wittrock,  1974 )  when  he  proposed  and  found  in  his  research  that 
highly  skilled  learners  are  more  likely  to  possess  and  spontaneously  use  generative  teaming  strategies; 
whereas  less  skilled  learners  are  more  likely  to  use  generative  straegies  only  when  directly  guided  to  do  so 
dunng  learning. 

2)  With  respect  to  the  content  characteristics,  the  results  revealed  that  ECS  was  effective  with  orgnaized 
and  familiar  content;  whereas  the  GCS  was  effective  with  random  and  unfamiliar  content.  Perhaps,  the 
explanation  of  these  results  has  to  do  with  learner  charactenstics.  We  mentioned  above  that  the  high  ability 
students  are  more  able  to  function  under  GCS;  whereas  the  low  and  medium  ability  are  more  able  to 
function  under  ECS.  The  high  ability  students  are  also  more  able  to  deal  with  unorganized  and  unfamiliar 
content  because  of  the  high  learning  ability  they  own.  And  the  organizational  skill  is  one  of  those  special 
abilities  that  the  distinguished  students  do  have,  so  they  can  apply  it  in  organizing  random  and  umfamiliar 
content;  whereas  the  low  and  medium  ability  students  are  most  of  the  time  tacking  in  such  skills,  thus,  they 
can  not  deal  with  random  and  novice  content  as  well  as  the  high  ability  students  can  do  ( Carter  &  Wittrock, 
1975.  Doctorow,  Wittrock,  &  Marks,  1978;  DIVesta  &  Peverly,  1984;  Lambiotte  &  Dansereau.  1992;  Sharger 
&  Mayer.  1989). 

3)  Considering  the  levels  of  learning,  the  results  also  revealed  that  ECS  was  more  effective  In  promoting  low 
and  medium  levels  of  learning;  whereas  GCS  was  more  effective  In  promoting  all  levels  of  learning:  low, 
medium,  and  high.  The  major  explanation  for  these  results  could  be  that  the  students  under  ECS  receive 
the  prepared  cognitve  activators  from  the  teacher  and  are  required  to  manipulate  them  only,  thus,  their  role 
dunng  teaming  is  supposed  to  be  passive;  whereas  students  under  GCS  are  required  to  generate  and 
manipulate  cognitive  acitvators,  thus,  their  rote  Is  expected  to  be  active.  Therefore,  It  la  obvious  that 
students'  involvement  in  generation  activity  makes  them  process  the  content  Information  on  a  semantic  and 
deep  level  of  learning;  whereas  students  who  are  not  involved  in  such  activities  and  Just  receive  prepared 
cognitive  activators  instead,  are  not  expected  to  process  information  deeply,  and,  to  retrieve  it  on  high 
levels  of  learning. 

This  rationale  was  supported  by  several  researcher  studies  suchs  as,  Dlnne!  A  Glover,  1985;  Peper  & 
Mayer,  1986;  Shrager  and  Mayer,  1989;  Weinstein  &  Mayer,  1985.  They  agreed  that  as  long  ?  s  students 
are  getting  actively  involved  in  learning  activities,  they  will  proccess  Information  on  a  deep  leve*  of  learning. 
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2-  Pre  vs.  Pott  Cognitive  Strategy  Activators'a  Position: 

The  review  of  previous  studies  on  pre  vs.  post  cognitive  strategy  activators  have  also  revealed 
important  results  with  its  relation  to:  1)  learner  characteristics.  2)  content  charactenstics,  and  3)  levels  of 
learning: 

1}  With  respect  to  the  learner  characteristics,  the  results  ot  this  research  indicated  that  pre  cognitive 
activators  were  more  effective  with  low  ability  students;  whereas  post  cognitive  activators  were  more 
effective  with  medium  and  high  ability  students. 

One  explanation  for  this  finding  is  that  the  selective  attention  function  of  pre  cognitive  activators  is  more 
important  for  low  ability  students  than  for  medium  or  high  ability  students;  whereas  the  memory  refreshing 
function  of  post  cognitive  activators  is  more  important  for  high  ability  students  than  for  low  ability  students 
( e.g.,  Prase,  1967;  1968;  Darwazeh,  1982;  Hudglns,  et  al.  1979;  Memory,  1983).  It  could  be  that  without 
selective  attention,  low  ability  students  cannot  remember  important  information  presented  in  the  passage. 
High  and  medium  ability  students,  on  the  other  hand,  can  remember  most  of  the  important  information 
without  the  aid  of  selective  attention  aids,  as  they  are  good  readers  and  have  a  better  background  in 
learning  content.  Therefore,  the  high  and  medium  ability  students  profit  more  from  a  memory  refresher, 
which  the  post  cognitive  activators  provide. 

2)  with  respect  to  the  content  characteristics  the  findings  of  this  research  revealed  that  pre  cognitive 
activators  were  more  effective  with  a  content  sequenced  form  specific  to  general  information;  whereas  the 
post  cognitive  activators  were  more  effective  with  a  content  sequenced  from  general  to  specific  information. 

One  explanation  of  these  results  could  be  that  when  instruction  begins  at  the  most  detailed  level, 
learners  need  to  start  with  a  synthesizer  that  is  represented  in  the  cognitive  activator  to  provide  context  for 
each  detailed  concept.  On  the  other  hand,  when  the  instruction  is  arranged  in  a  general-to-detailed 
sequence,  it  would  appear  that  the  most  general  concepts  themselves  provide  the  context  for  the 
subsequent  concepts,  such  that  students  do  not  benefit  from  a  pre  CSA  as  an  initial  overview  of  the  set  of 
concepts  (e.g. ,  Prey  and  Reigeluth.  1981 ;  Sanders,  1973).  Conversely,  in  the  general  to  detailed 
sequence,  students  appear  to  need  a  synthesizer  that  is  representd  in  the  cognitive  activator  at  the  end  of 
the  instruction  to  review  the  relationships  among  the  concepts 

3)  With  respect  to  levels  of  learning  the  findings  also  revealed  that  pre  cognitive  activators  were  more 
effective  in  promoting  low  levels  of  learning  such  as  memory,  intentional,  or  direct  levels;  whereas  post 
cognitive  activators  were  more  effective  in  promoting  high,  incidental,  or  indirect  levels  of  learning.  One 
explanation  of  this  finding,  as  was  supported  by  Darwazeh  &  Reigeluth  (1982);  Andre,  et  al.  (1981)  In  their 
experiment  No.1);  Anderson  &  Biddle  (1975);  Rothkopf  &  Blsbicos  (1967);  Sanders  (1973)  is  that  pre 
cognitive  activators  serve  as  a  method  of  of  arousing  ■  selective  attention4.  Hence,  while  reading  the 
passage,  the  (earner  will  focus  attention  on  those  thoughts  and  ideas  which  are  related  directly  to  the  pre- 
cognitive  activators  and  will  neglect  the  ideas  and  thoughts  which  are  not  related  to  the  pre  CA.  However 
because  post  CA  comes  after  the  passage,  they  cannot  serve  as  a  selective  attention  function;  rather  they 
can  only  serve  to  refresh  the  learner's  memory  within  and  beyond  the  frame  of  the  passage,  that  is  they 
reinforce  intentional  and  incidental  levels  of  learing  ( e.g.,  Andre  &  Womack.  1978;  Darwa2eh  &  Reigeluth. 
1 982).  Well,  further  future  studies  are  recommended  on  this  issue. 

3-  Visual  vs.  Written  Cognitive  Strategy  Activators: 

The  review  of  previous  studies  on  visual  vs.  written  cognitive  strategy  activators  have  reveaiedthe 
following  findings: 

1 )  With  respect  to  learner  characteristics  the  previous  studies  revealed  that  visual  (concrete)  CSA'S  were 
more  effective  with  low  ability  students;  whereas  the  written  (abstract)  CSA's  were  more  effective  with  high 
ability  students  (  e.g.,  Holmes,  1987;  New&  Stepich,  1991 ;  Winn  &  Sutherland,  1989;  Cha  &  Owyer,  1991). 
The  most  reasonable  explanation  for  these  findings  could  be  that  students  using  the  visual  CSA's,  depend 
on  more  than  one  sense  to  manipulate  them.  They  could  use  pictures,  graphic  organizers,  maps  or  other 
concrete  CSA  beside  reading  the  passage.  Thus,  using  more  than  one  sense  while  manipulating  the  CA 
will  lead  to  a  better  understanding,  hence  to  better  learning.  In  addition,  using  visual  or  concrete  CSA  will 
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simplify  the  complicated  ideas  mat  are  presented  in  the  passage.  Therefore,  this  will  help  low  ability 
students  in  particular,  who  lack  ot  the  ability  to  deal  with  complex  and  abstract  CSA'S  like  the  written  ones, 
more  than  high  ability  students  who  have  the  ability  to  deal  with  Complex  and  abstract. 

On  the  other  hand,  visual  CSA's  usually  summarize  the  content  and  its  relationships  and  present  them 
in  f iguree,  diagrams,  maps,  trees,  (low  charts,  etc..  This  would  help  low  ability  students,  who  find  It  difficult  to 
skim  the  content  and  see  its  relationships,  to  pay  attention  to  the  essential  ideas,  hence  to  understand  the 
learning  content.  The  opposite  is  true  in  the  case  of  using  written  CA  which  use  the  narrated  language  for 
this  purpose.  Therefore,  only  the  high  ability  students,  who  can  deal  with  the  abstract  language,  can  befifit 
from  this  kind  of  activators. 

2)  With  respect  to  the  content  characteristics,  the  general  trend  of  previous  studies'  results  revealed  that 
the  visual  CSA's  are  more  effective  with  unfamiliar  content;  whereas  the  written  CSA's  are  more  effective 
with  the  familiar  ones  ( e.g.,  Winn  &  Sutherland,  1989;  Lambiotte  &  Dansereau,  1992,  Lambiotte,  1993). 
According  to  our  above  explanation,  two  characteristics  of  visual  CSA  are.  the  concreteness  and  clearness. 
Such  characteristics  are  expected  to  make  the  unfamiliar  content  familiar,  hence,  more  manageble  and 
understandable,  in  contrast,  two  characteristics  of  a  written  CSA  are  abstraction  and  complex.  Such 
characteristics  are  not  expected  to  make  the  content  familiar,  manageble,  or  understandable,  because, 
these  CSA's  need  some  clarification  themselves.  So,  Students  in  this  case  would  not  get  the  optimal 
benefit  from  such  activators  unless  they  were  intelligent  enough  to  deal  with  the  abstract  and  symbolic 
CSA's.  Thus,  further  experiments  are  recommended  on  this  matter. 

3)  Finally,  with  respect  to  the  levels  of  learning,  the  research  revealed  that  the  visual  CSA's  are  likely  to  be 
more  effective  in  promoting  low  or  sometimes  medium  levels  of  learning;  whereas  the  written  CSA's  are 
likely  to  be  more  effective  in  promoting  high  levels  of  learning  (e.g..  CorMII.  et  al.  1968;  New  &  Stepich. 
1991).  The  results  make  sense  in  terms  of  the  level  of  processing  that  the  CSA's  needed.  The  visual 
CSA's  most  of  the  time  are  simple,  concrete,  and  clear,  so,  they  are  easier  to  be  understood  without  the 
need  of  deep  thinking.  On  the  other  hand,  the  written  CSA's  most  of  the  time  are  complex,  abstract,  and 
not  clear,  so.  they  need  from  the  students  a  great  deal  of  attention  and  deep  processing  of  information  in 
order  to  manipulate  and  understand  them .  The  deep  processing  mostly  leads  to  a  high  level  of  learning 
such  as  application  and  problem  solving,  (or  Incidental  levels);  whereas  the  shallow  processing  of 
information  mostly  lead  to  a  shallow  level  of  learning  such  as  remember  (or  intentional  levels).  So,  this  could 
be  a  reason  that  the  visual  CSA's  promote  low  or  medium  level  of  learning;  whereas  the  written  CSA's  lead 
to  a  high  level  of  learning.  Further  experiments  are  recommended  on  this  matter  too. 

Research  Implamentlon 

The  research  findings  havean  important  implemention  for  designing  instruction  based  on  cognitive 
strategy  activators. 

The  implemention  of  this  research  is  represented  in  proposing  two  prescriptive  models  for  designing 
instruction.  These  two  models  depend  basically  on  Dick  and  Carey's  model  for  designing  Instruction  ( Dick 
4  Carey,  1 990).  The  first  model  is  for  embedded  cognitive  strategy  activators,  and  the  second  model  is  for 
generative  cognitive  strategy  activators.  These  models  are  recommended  to  be  used  by  teachers  when 
they  plan  for  every  day  teaching,  and  by  instructional  designers  and  developers  when  they  design  schools' 
curriculum,  computers1  programmes,  or  business  projects.  These  models  will  be  discussed  from  four 
angles:  1 )  Rationale  of  creating  such  models,  2)  their  outcomes,  3)  method,  and  4)  condrttone. 

1-  Rationale; 

The  Science  of  Instructional  Design  became  prosperous  In  the  last  two  decades  in  different  countries 
of  the  world.  Accordingly,  instructional  designers  have  done  a  great  Job  in  designing  instruction  based  on 
the  pnnciples  of  Instructional  design  science  ( e.g.;  Briggs,  1977;  Briggs  A  Wager,  1961;  Darwazeh.  1966; 
Dick  &  Carey.  1990;  Gagne.  Briggs.  &  Wager,  1992;  MlrreN.  1  Tenyseon.1977;  Reigeiuth,(Ed.)  1983; 
Snelbeker,  1974;  Wager,  et  al.  1990).  But  still,  the  cognitive  strategy  activators  are  not  considered  in  their 
models. 
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Recently,  thwe  was  an  attempt  by  West  and  his  colleagues  to  specify  a  template  for  designing 
instruction  based  on  cognitive  science  principles  ( West,  Farmer .  &  Wolf,  1992,  pp.  209-263)  but  still  they 
did  not  specify  a  clear  procedure  on  how  to  design  instruction  with  a  consideration  of  cognitive  strategies. 

Therefore,  the  major  purpose  of  the  current  research  is  to  propose  two  prescriptive  models  for 
designing  instruction:  one  model  is  for  Embedded  CSA's,  and  the  another  model  is  for  Generative  CSA's. 
1  believe  that  without  considering  the  embedded  and  generative  cognitive  strategy  activators  while 
designing  instruction,  the  learning  and  instructional  process  will  be  incomplete  specially  in  this  decade 
which  is  accompanied  by  cognitive  revolution  in  the  field  of  psychology.  Stimulating  students1  cognitive 
strategies  has  become  a  very  important  issue  for  enhancing  and  accelerating  their  learning. 

2-  Outcome*: 

Using  embedded  or  generative  cognitive  strategy  activators  under  certain  conditions  will  help  learners, 
teachers,  devolopers,  and  instructional  designers  to  do  the  following: 

a)  For  Learners: 

1  -  Stimulate  their  cognitive  strategies  and  use  them  properly  during  learning. 

2-  Focus  attention  on  the  content  of  instructional  materials,  hence  increase  their  level  of  motivation  to 
learning. 

3-  input,  process,  and  output  information  effectively. 

4-  Create  linkages  between  the  existing  knowledge  and  the  new  one,  and  between  the  ideas  in  the 
text  itself,  hence  making  the  content  more  manageble  and  understandable. 

5-  Simplify  complicated  information  to  be  learned,  and  make  it  more  managable,  and  understandable. 

6-  Process  information  on  deep  (or  high)  level  of  learning,  hence  to  store  it  in  long  term  memory. 

7-  Retrieve  information  for  later  use  specially  for  problem  solving  either  for  solving  academic  or  persoal 
problems. 

b)  For  Teec    %  Developers,  and  Instructional  Designers: 

1-  Design  instruction  comprehensively  and  integrated. 

2-  Compensate  the  defect  In  teaching  method,  if  any,  specially  for  novice  teachers. 

3-  Compensate  the  defects  in  the  organization  of  the  learning  content  ( i.e.,  incoherent,  unfamiliar,  or 
random  content )  and  make  it  coherent,  familiar,  and  organized  as  much  as  possible. 

4-  make  the  learning  content  more  interesting. 

5-  Promote  learning  on  high  levels  rather  on  low  levels. 

6-  Enhance  the  quality  of  instructional  process  generally. 

3-  method  (how  to  use): 

1-  Establish  overall  goals  ( ECS  &  GCS  models). 

2-  Conduct  content  analysis  and  determine  Its  characteristics.  Is  it  organized  or  unorganized;  familiar  or 
unfamiliar;  sequenced  from  specific-  to-general  or  from  general-to-specific  information;  containes  mainly 
concepts,  principles,  procedures,  or  tacts;  long  content  or  short  one,  or  what  I  determine  (ECS  &  GCS 
models). 

3*  Conduct  learner  analysis  and  determine  If  they  are  above  average,  average,  or  below  average;  field- 
dependent  or  field-independent;  high  motivated  or  low  motivated;  trained  or  untrained  learners,  or  what  I 
(ECS*  GCS  models). 

4*  identify  the  level  of  performance  that  you  want  students  to  demonstrate  after  instruction.  Is  it  a 
knowledge  (low)  level,  or  comprehension  (medium)  level,  or  application,  analysis,  synthesis,  problem 
solving,  evaluation  (high)  level  of  learning,  or  what  I  determine  (ECS  &  GCS  models). 

5-  Develop  test  items  for  measuring  each  level  of  performance  that  you  have  determined  (ECS  &  GCS 
models). 


r  )f%  920 


6-  Develop  or  select  instructional  materials  and  mass  rredir>  ti  tat  are  suitable  for  teaching  the 
determined  teaming  content  (ECS  &  GCS  models). 

7-  Develop  instructional  strategies  for  teaching  each  component  of  the  content  that  you  have 
determined  (ECS  &  GCS  models). 

8-  Develop  cognitive  strategy  activators  to  be  used  (ECS  model). 

9-  Develop  directions  or  a  training  program  for  generating  cognitive  strategy  activators  ( GCS  model). 

10  -  Determine  the  kind  of  cognitive  strategy  activators  to  be  used.  Is  it  adjunct  questions,  advance 
organizers,  note-taking,  pictures,  outlines,  stories,  underlining,  etc...  or  what !  (ECS  &  GCS  models). 

1 1  -  Determine  the  position  of  the  cognitive  strategy  activators  to  be  used.  Is  it  pre,  during,  or  after  the 
passage !  (ECS  &  GCS  models). 

12-  Determine  the  mode  of  cognitive  strategy  activators  to  t:t  used.  Is  it  concrete  or  abstract,  audio- 
visual or  written  »  (ECS  &  GCS  models). 

13-  Design  and  conduct  formative  evaluation  (ECS  &  GCS  models). 

1 4-  Revise  instruction  according  to  the  results  of  formative  evaluation  (  ECS  &  GCS  models). 

1 5-  Design  and  conduct  summative  evaluation  (  ECS  &  GCS  models). 

 Insert  Figure  4  about  here  

*-  Conditions  (When  to  use): 

1-  Use  (ECSM)  if  you  deal  with  low  or  medium  ability  students;  otherwise,  use  the  (GCSM)  when  you 
deal  with  high  ability  students. 

2-  Use  (ECSM)  if  you  have  students  charactarized  by  field-dependent  cognitive  style;  otherwise,  use 
(GCSM)  when  you  have  students  charactarized  by  field-Independent  cognitive  style. 

3-  Train  Students  if  you  use  (GCSM)  only,  unless  students  under  (ECSM)  faild  to  manipulate  the  given 
cognitve  activators. 

4-  Use  (ECSM)  if  you  deal  with  organized  content  of  learning;  otherwise,  use  (GSM)  when  your  deal  with 
unorganized  or  random  content  of  learning. 

5-  Use  (ECSM)  if  you  deal  with  familiar  content  of  learning;  otherwise,  use  (GCSM)  when  you  deal  with 
the  unf  ami  liar. 

6-  Use  (ECSM)  If  you  want  learning  to  occur  on  low  or  medium  levels;  otherwise  use  (GCSM)  when  you 
want  learning  to  occur  on  all  levels  of  learning  ( Go  back  to  Bgure  1). 

7-  Under  (ECSM).  Use  pre-coc nitive  strategy  activators  h*  you  deal  with  low  ability  students;  whereas  use 
post-cognitive  activators  when  you  deal  with  either  medium,  or  high  ability  students.  But  under  (GCSM), 
use  cognitive  strategy  activators  during  or  after  the  passage,  with  any  level  of  students  ability  specially  with 
high  abilities,  and  for  any  level  of  learning  specially  for  high  levels. 

8-  Under  (ECSM),  use  pre-cognittve  strategy  activators  if  you  have  a  content  sequenced  from  specific 
to  general  information;  whereas,  use  post-cognitlva  strategy  activators  if  you  have  a  content  squenced  from 
general  to  specific  information.  But  under  (GCSM)use  the  oognitlve  strategy  activators  during  or  after  the 
content  with  any  type  of  Its  sequence. 
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9*  Under  (ECSM),  us*  pre-cognitive  strategy  activators  it  you  want  learning  to  occur  on  low  levels  such 
as  remember,  intentional,  or  direct  teaming;  whereas  use  post-cognitive  activators  K  you  want  learning  to 
occur  on  all  levels  of  teaming  such  as  comprehenron,  application,  analysis,  synthesis,  evaluation,  problem 
solving  But  under  (QCEM)  use  during  or  post-cognitive  strategy  activators  tor  promoting  any  level  of 
learning  specially  for  high  levels  (Go  back  to  Figure  2). 

10*  Use  visual  (concrete)  cognitive  strategy  activators  if  you  have  low  ability  students,  whereas  use 
written  (abstract)  cognitive  strategy  activators  if  you  have  high  ability  students  (ECS  &  GCS  models). 

1 1-  Use  visual  (concrete)  cognitive  strategy  activators  H  you  have  unfamiliar  content,  whereas  use 
written  (abstract)  cognitive  strategy  activators  if  you  have  familiar  content  (ECS  &  GCS  models). 

12-  Use  visual  (concrete)  cognitive  strategy  activators  it  you  want  teaming  to  occur  on  low  or  medium 
levels,  whereas  use  written  (abstract)  cognitive  strategy  activators  if  you  want  learning  to  occur  on  high 
levels  (ECS  &  GCS  models),  (  Go  back  to  Figure  3). 

Recommendations 

Considering  the  importance  of  cognitive  strategy  activators  in  promoting  and  enhancing  students1 
learning  in  particular,  and  teachers'  teaching  in  general;  the  potential  for  continuing  and  extending  research 
on  the  effect!  of  generative  activities  is  very  crucial,  Given  past  research  results,  several  suggested  studies 
are  recommended  on  cognitive  strategy  activators  to  compensate  the  shortage  in  the  previous  studies. 
Future  studies  are  recommended  to  manipulate  the  folloing  issues: 

1-  Comparing  ECS's  with  GCS's  by  using  other  kinds  of  cognitive  activators  that  were  rarely  used  in  the 
past,  such  as:  abstracts,  objectivs,  stones,  analogies,  synthesizers,  and  mnemonic  devices  (  Carney,  et  al., 
i988;Pressley,  et.  al.,  1976). 

2-  Using  posttest  measuring  high  levels  of  learning  such  as  analysis,  synthesis,  evaluation,  and  discovery 
levels  beside  the  remember  and  comprehension  levels  (Bloom,  1956), 

3-  Using  different  lengths  of  learning  content :  short,  medium,  and  long  content  (  Retgeluth,  1961). 

4-  Using  different  modes  of  cognitive  strategy  activators  such  as  abstract  vs  concrete;  audio  vs.  visual  vs. 
written. 

5-  Using  different  learners  characteristics  such  as  high  anxiety,  low  anxiety,  high  motivated,  lowe motivated. 
Plus,  using  subjects  in  different  stages  of  education  such  as  kindergarten,  elementary,  middle,  scondary, 
and  graduate  stages. 

The  aim  of  experimenting  on  such  issues  is  to  help  researchers,  hence,  instructional  designers  to 
specify  more  conditions  for  using  the  cognitive  strategy  activators.  Therefore,  they  can  modify  or  add  to  the 
above  suggested  models  whenever  necessary. 

Summary: 

The  research  tried  to  accomplish  three  ends: 

1)  give  definitions  and  clarifications  on  cognitive  strategy  activators  in  terms  of  their  system,  positions,  and 
modes. 

2)  Summarize  a  sample  of  the  previous  experimental  and  theoretical  studies  which  were  conducted 
between  ;960's  -  1990's  on  cognitive  strategy  activators  In  terms  of  their  systems,  positions,  and  modes, 
and  with  their  relation  to  three  instructional  variables:  a-  learner  characteristics,  b-  content  characteristics, 
and,  c-  levels  of  learning. 


3)  Propose  two  instructional  design  models  based  on  cognitive  strategy  activators:  one  (or  embedded 
cognitive  activators  in  which  teachers,  developers,  or  instructional  designers  are  expected  to  take  the  major 
role  in  generating  the  cognitive  activators,  and  the  other  model  (or  generative  cognitive  activators  in  which 
learners  are  expected  to  take  the  major  role  in  generating  such  activators. 

These  two  proposed  models  are  expected  to  lay  a  foundation  of  a  theory  on  how  to  desgin  instruction 
with  a  consideration  of  cognitive  strategy  activators.  At  the  same  time,  they  are  considered  as  an  invitation 
for  researchers  to  conduct  more  studies  on  this  subject,  with  the  aim  of  increasing  the  validity  of  those 
models  which  was  extracted  from  previous  studies,  hence,  to  modify  and  add  to  them  whenever  necessary. 


REFERENCES 


Ally.  M..  &  Szabo,  M.  (1992).  Effects  of  students  generated  vs.  instructor  provided  spatial  strategy  on 
teaming  from  text.  Paper  Presented  at  the  Annual  Meeting  of  the  Association  for  Educational 
Communications  and  Technology:  (AECT),  Washington,  D.C.  (Feb.5  -Feb. 9. 1992). 

Alexander.  L,  Frankiewicz.  R.  G.,  &  Williams.  R.  E.  (1979).  Facilitation  of  learning  and  retention  of  oral 
instruction  using  advance  and  post  organizers.  Journal Qf  Educational  Psychology.  71(5),  701-707. , 

Andre.  M.  E.,  &  Anderson,  T.  H.  (1987-1979).  The  development  and  evaluation  of  a  self  questioning 
technique.  Reading  RftfiAarnh  Qnartarly  14(4),  605-623. 

Andre.  T .  Muller,  C.  Womack.  S..  Smid,  K.,  &  Tuttle,  M.  (1980).  Adjunct  application  questions  facilitate 

later  application  or  do  they? Journal  of  Education  Psychology,  72(4),  533-543. 
Anderson.  B.  W.  (1973),  The  effect  of  pre-  versus  post-organizers  upon  retention  of  economic  concepts  at 

the  collegiate  level.  Pupation  Abstract  International  1973.  34.  381 9A. 
Anderson.  R.  C .  &  Biddie,  W.  B.  (1975).  On  asking  people  questions  about  what  they  are  reading.  In  G 

Bower.  (Ed.),  Psychology  of  Learningjtnd  Motivates.  Vol.  9.  New  York:  Academic  Press. 
Anderson.  R.  C  ,  A  .Kulhavy,  R.  W.  (1972).  Imagery  and  prose  learning  Journal  of  Educational  Psychology 

63(3),  242-243. 

Anms.  L.  F.  (1 965).  Students  generated  paragraph  summaries  and  the  information  processing  theory  of 

prose  learning.  JoumaLof  Experimental  Education,  54(1),  4-10. 
Ausubel.  D.  P.  (1960).  The  use  of  advance  organisers  in  the  learning  and  retention  of  meaning  for  verbal 

learning  material  Journal  of  Fdtigatinpal  Psychology ,  51(5),  267-  272 
Ausubel.  D.  P.  (1964).  Some  psychological  aspects  of  the  structure  of  knowledge,  in  S.  Elam  (Ed.), 

Educational  and  structure  nf  knowledge.  U.S.A.:  Rand  McNaily. 
Ausubel.  O  P.  (1966).  Educational  pshchology:  A  cognitive  vaiew  New  York:  Holt,  Rinehart.  &  Winston 
Ausubel.  D.  P.,  &  Fitzgerald,  D.  (1962).  Organizer,  general  background  and  antecedent  learning  variables 

in  sequential  verba]  learning.  Journal. of  fducational  Psychology,  53(6).  243*249. 
Ausubel  D  P.,  &  Youssef.  M.  (1963).  The  role  of  discriminability  in  meaningful  parallel  leamingJournal  of 

Educational  Psychology,  54,  331-336. 
Babbs.  P..  &  Moe,  A.  (1983).  Metacogntion:  A  key  for  independent  learning  from  text.  Ifae.Heading 

Ieacher.  36(4).  422-426. 

Barnes.  B.  R.(  &  Claw  son.  E.  U.  (1975).  Do  advance  organizers  facilitate  learning?  Recommendations  for 

further  research  based  on  an  analysis  of  32  studies  Revifiw  of.  Educational  .Research  45(4),  637-659. 
Barron,  R.  R.  (1971).  The  effects  of  advance  organizers  upon  the  reception  learning  and  retention  of 

general  science  content.  ERIC  Document  ED:  061-554 
Bauman,  D.  J.,  Glass.  G.  V.,  &  Harrington,  S.  A.  (1969).  The  effects  of  position  of  an  organizers  of  learning 

meaningful  verbal  materials  Fte&earch  Payw  Nn  ?d  i  ahnratnry  q|  phi  iftatigpai  R^arph  University 

of  Colorado. 

Barnett,  J.  E,  DiVesta,  F.  J.,  &  RogozensW,  J.  T.  (1981).  What  is  learned  in  note  taking  Journal  of 

Educational  Psychology,  73(2).  181-192. 
Barry.  R.J.  (1974).  The  concept  of  mathemaginic  behaviors:  An  analysis  of  its  heuristic  valuePerceptuaL 

and  MototSMts  36.  311-321. 
Bernstein,  S  L  (1973).  The  effect  of  children's  questions  asking  behavior  on  problmn  solution  and 

comprehensjn  of  written  material.  Dissertainn  Abstract  International  34,  3130A. 
Beentjes,  J.  W,  &  Van  der  Voot,  T.  H.  (1992).  Children's  written  accounts  of  televised  and  printed  stones 

ETRAD.  39(3),  15-26 

Blanchard,  J.  8.  (1985).  wwhat  to  tu  students  ehout  underlining  and  *hy*inum«i  of  Riding  (rw )  199. 

202. 

Blanchard.  J  S  &  Mik!;elson,  V.  (1987).  Underlining  performance  outcomes  in  expository  text.  Journal  nf 

Educational  Research  80(4),  197*201. 
Bloom,  B.  S.  (Ed.).  (1956).  laxannmiea  nf  iducatinnal  nhjnntivftt  UandbooiLlL_Cognitive  Domain  NY: 

McKey. 

Bretzing,  B  H.(  Kulhavy,  R.  W.,  &  Catrinor  L.  C.  (1987).  Notetaklng  by  Junior  high  students  Journal  of 


Educational  fiesearch.  80(6).  359-362. 
Bruner.  J.  S.  (1960).  TJia.pfflcess.ot  education .  New  York:  Random  House. 
Bull;  B.  L.  &  Wittrock.  M.  C.  (1973).  Imagery  in  the  leamigt  of  verbal  definition  flnftsb  Journal  of 

EducationalPsyctalogy.  43(3),  289-293. 
Carney.  R.  N..  Levin.  J.  R.,  &  Morrison.  C.  R.  (1988).  Mnemonics  learning  of  artists  and  their  paintings. 

Amancan  Educational  Research  Journal  25(1),  107-125 
Garner,  C.  Joseph,  M.  R.,  Krey,  C,  &  Lacroix,  P.  (1963a).  Supplied  Visuals  and  imagery  instruction  in  field 

independent  and  field  dependent  childnYs  recall.  Pr*i  rational  Communication  and  Technology 

Journal.  31(3),  153-160. 

Carrier.  C.  Karbo.  K..  KSndem,  H.,  Legisa.  G.,  &  Newstrom,  L  (1983b).  Use  of  self- Generated  and  supplied 
visual  as  mnemonics  in  gifted  childrens  learning.  Perceptual  and  Motor  Skills  57(1).  235-240. 

Carrier.  C.  Davidson.  G..  Higson,  V..  &  Williams.  M.  (1984).  Selection  of  options  by  field  independent  and 
field  dependent  children  in  a  computer  based  concept  lesson.  Journal  of  Cnmnt^r  instruction 
11(2).  49-54. 

Carrier.  C.  Davidson.  G..  &  Williams.  M.  (1985).  The  selection  of  instructional  options  in  a  computer  based 
coordinate  concept  lesson.  Educational  Communication  and  Technology  Journal  33(3)  199-212. 

Carson.  H.  C.  &  Reigeluth.  C  M.  (1983).  The  effects  of  sequence  and  synthesis  on  concept  learning 
using  a  parts  conceptual  structure.  IDiS&EJA&cking-EdpeLM^  Syracuse  University,  Syracuse,  NY. 
13244-2340. 

Cha.  D..  &  Dwyer.  F.  M.  (1991).  Effect  of  visual  and  embedded  rehearsal  strategies  in  facilitating  students 
achievement  of  different  educational  objectives.  Paper  Presented  at  the  Annua!  Meeting  of  the 

Ayayiatjpn  for  Frinrafinnal  Primmi mirations  and  Technology  (ARTP  Ortano,  Florida  (13-17,  1991). 

Chao,  C.  I..  &  Reigeluth.  C.  M.  (1986).  The  effects  of  format  and  structure  of  a  synthesizer  on  procedural 
decision  learning.  iDDftF  Working  Paper  No     Syracuse  University.  Syracuse  ,  NY.  13244-2340. 

Clarv..  R.  E.  (1  ->9).  The  future  of  technology  in  deucational  psychology.  In  M.C.  Wittrock.  4  F.  Farley 
Eds.).  ThP  future  of  educational  psychology  Hillsdaled.  NJ:  Lawrance  Erlbaum  Associates. 

Corkiil,  A.  J.,  &  Golver,  J.  A.  (1988).  Advance  organizers:  Concrete  versus  abstract  Journal  of  Education^ 
Research  82(2),  76-81. 

Grouse.  J.  H..  &  Idesein.  P.  (1972).  Effects  of  encoding  cues  on  proselearning.  Journal  of  Educational 

Psychology ,  63(4).  309-313. 
Curtis.  R.  V..  &  Reigeluth,  C.  M.  (1984).  The  use  of  analogies  in  written  text  inataictioiial  Science,  13, 

99-117. 

Dansereau.  0.  F..  Collin-  K.  W..  McDonald.  B.  A.,  Holley,  C.  D..  Diekhoff.  G.,  &  Evans.  S.  H.  (1979). 
Development  and  evaluation  of  an  effective  learning  strategy  training  program  JournaJ  of  education 
Psychology.  71.  64-73, 

Darwazeh,  A.  N.  (1991).  Cognitive  strategy  activators:  Instructional  aids  for  inhancing  learning  and 

instruction.  Bethlehem  University  Journal.  10. 41-77. 
Cc^azeh,  A.  N.  (1992,1993).  Summaries  of  exoenmenta1  stupes  of  embedded  versos  detacned 

-sgr.itive  strategy  activators.  Two  Proocsed  Instructional  MedeisAre  Suomitted  to  Experimentation. 

Bethlehem  University  Journal,  11.12.  29-85 
Darwazeh,  A.  N.  (1988).  Advance  organizers  as  embedded  and  detached  cognitive  strategy  activators: 

Their  effects  on  remember  and  application  -  level  leamingThn  Arabic  Journal  of  Research  for  Higher 

Education.  (December,  1988).  3-44,  Damascus,  Syria. 
Darwazeh,  A.  N.  (1982).  Student  generated  versus  teacher  generated  adjunct  questions:  Their  effects  on 

remember  and  application-  level  learning.  ni*x*rtatiftn  Abstract  international  47(07),  2281 A.  (Jan., 

1983). 

Darwazeh.  A.  N..  &  Reigeluth,  C.  M.  (1982).  Type  and  position  of  adjunct  questions:  The4r  effects  on 

memory  and  application.  ID04E  WorKingn  Paper  No.  7,  Syracuse  University,  Syracuse,  NY.  13244- 

2340.  FRin  Document*.  ED:  288-517. 
Darwazeh.  A.  N.  (in  press).  Embedded  vs.  specified  and  unspecified  generative  cognitive  strategy 

activators:  Their  effects  on  memory  and  application  levels  teaming  An  Nap^ionmai  m  p»««arr*)  (a), 

Nablua.  West-Bank. 

Dean.  T.  W.,  Sea/les,  D.,  Singer.  H.,  &  Cowen,  S.  (1990).  Learning  concepts  from  biology  text  through 
K      pictorial  analogies  and  an  analogical  study  guide.  Journal  of  Eriimallnal  Rpfyarcft  83(4).  233-237. 
Dee-Lucas.  D..  &  DiVesta.  F.  J.  (1980).  Learner  generated  organizational  aids:  Effects  on  learning  from 
text.  Journal  of  Educational  Psychology,  72(3),  304-311. 


Dick,  W.,  &  Carey.  L  (1990).  The  systematic  design  of  instruction  (3ed  ed.).  U.S.A.:  Scott,  Foresman  and 
Company. 

Dinnel,  D..  &  Glover.  J.  A.  (1985).  Advance  organizers:  Encoding  manipulations  Journal  of  Educational 
Psychology.  77(5).  514-521. 

DIVesta.  F.  J.  (1969).  Applying  cognitive  psychology  to  education.  In  M.  C.  Wrttrock,  &  F.  Farley  (Eds.), 
The  future  of  educational  psychology  Hillsdaled,  NJ:  Lawrence  Erlbaum  Associates. 

DiVesta.  F.  J.,  &  Pevertey.  S.  T.  (1984).  The  effect  of  encoding  variability,  processing  activity,  and  rule- 
example  sequence  on  the  transfer  of  conceptual  rules.  Journal  of  Educational  Psychology,  76(1 ). 
108-119. 

Davey.  B.,  &  McBride.  S.  (1986).  Generating  self-questions  after  reading:  A  comprehension  assist  for 

elementary  students.  Journal  of  Educational  Psychology,  80(1).  43-46. 
Duell.  0.  K.  (1 977).  Overt  and  covert  use  of  objectives  of  different  levels.  Paper  presented  at  the  Annual 

Meeting  ol  the  American  Educational  Hesearch  Association  (  New  York,  Ny.,  1977). 
Ooctorow,  M.,  Wittrock.  M.  C.  ft  Marks.  C.  B.  (1978).  Generative  processes  in  reading  comprehension. 

Journal  of  EducationaLPsychology,  70(2).  109-118. 
Drane.  M  J  ,  Halpin.  G  M.,  Halpin,  W.  G..  Von  Eschenbach.  J.  F  .  &  Worden.  T.  W.  (1989).  Relationships 

between  reading  porticiency  and  field  dependence  /  field  independence  and  sex.  Educational 

Research  Quarterly  13(2).  2-9. 
Dreher.  M  J .  ft  Gambreli,  L.  B.  (1982).  Training  children  to  use  a  self-questioning  strategy  lor  studying 

expository  prose  Paper  presented  at  the  Annual  Mating  oi  the  American  Friitrationai  Baaaaffib 

Association  .(Mar  18-23. 1982).  New  York.  NY. 
Essen.  G.  V,  ft  Hamaker,  C.  (1990).  Using  self-generated  drawings  to  solve  arithmetic  word  problems. 

Journal  of  Ediiratinnal  Rfinnarch  83(6).  301-312. 
Feiker.  F,.  &  Dapra.  R.  A.  (1975).  Effects  of  question  type  and  position  placements  on  problem  solving 

ability  from  prose  material.  Joumal.of  Educational  Psychology,  67(3),  380-364. 
Prase,  L  T  ,  i  Schwartz.  B.  H.  (1975).  Effect  of  question  production  answering  on  prose  recall  Journal  of 

Educational  Psychology  67  ,  628-635. 
Frase.  L  T.  (1987).  Learning  from  prose  materials:  Length  of  passage,  knowledge  of  results,  and  positions 

ol  questions.  .Journal  of  Educational  Psychology.  67(3).  380-384. 
Prase,  L  T.  (1968)  Learning  from  prose  material:  Length  of  passage,  knowledge  of  results,  and  positions 

Of  questions.  JffMP1?!  nf  ^^t\nna\  Pgynhnlngy,  58(6).  266-272. 

Prase.  L  T.  (1 970).  Boundary  conditions  for  mathemagenic  hghavinre  Review  of  Frii^atirtnai  Research 
40,  337-348. 

Prey.  L.  &  Reigeluth.  C.  M.  (1981).  The  use  of  sequence  and  synthesis  for  teaching  concept.  IDD&E 

working  Paper  No  a  Syracuse  University.  Syracuse.  NY,  13244-2340. 
Gagne.  R.  M..  Briggs,  L.  J..  &  Wager,  W.  W.  (1992).  Principles  of  instructional  design  (4th  ed.).  U.S.A.: 

Holt.  Rinehart  &  Winston. 

Qrabowski,  B.  L  (1989).  Mathemagenic  effects  and  generative  learning  strategy  for  computer  based 
interactive  video,  unnupiifihad  Mflmi«rript  iddae  working  BapaiSyracuse  University,  NY, 
13244-2340. 

Harris,  B.  R.  (1992).  The  effects  of  integrating  a  learning  strategy  in  CBI.  Paper  Presented  at  the  Annual 
Meeting  of  the  Atiyi^  y  PHtir^innai  nnmmiiniritinnt  mmi  T^hnningy  (AECT),  Washington. 
D.C.(Feb.5-Feb.9. 1992). 

Helfeidt.  J.  P.,  &  Lalik.  R.  (1975),  Reciprocal  student-teacher  questioning.  The  ReadingBesearch.  30. 
283-287. 

Holmes.  B.  C.  (1987).  Children's  inferences  with  print  and  pictures  Jnnmp1  n1  Fr^Pthnal  Psynhnlnpy 
79(1),  14-18. 

Hoper.  S.f  Sales,  G ,  ft  Rysavy,  0.  (1992).  Generative  learning  In  small  groups.  Paper  presented  at  the 
Association  fry  Fry^^on"'  CoiununlcatiOM  anH  T^nningy  (AECT),  Washington,  D  C  (Feb.5  • 
Feb.9, 1992). 

Hudgins.  B.  B.,  Dcrman.  J.,  ft  Harris,  M.  (1979).  Some  affects  of  adjunct  questions  on  Intermediate  grade 
children  with  differing  reading  comprehension  abilities,  ^'ircH  nt  Frtiinallnn*1  Q«M*rrh  72(5),  259- 
265. 

idstein,  P.  (1972).  Underlining  versus  repetitive  reading.  The  Journal  of  EducattanaLBaaeatcfa  65(7), 
321-323. 

Jonassen.  D,  K,  Hartley,  J.,  ft  Trueman.  M.  (1985).  The  effects  of  learner  generated  versus  experimenter 


t  i  r  926 


provided  headings  on  immediate  and  delayed  recall  and  comprehesion.  pair:  nnrnmgnts,  ED:  254- 
567 

Jonassen.  D.  H.  (1988)  Integrating  learning  strategies  into  courseware  to  facilitate  deeper  precessing.  In 
0  H  Jonassen  (Ed.).lnstfuctiQnaLdesi8DjQi  microcomputer  coiuiseware  Hillsdale,  NJ:  Lawrence 
Erilbaum  Associates. 

Jonassen.  D.  H.  Cole,  P..  &  Bamford.  C.  (1992).  Learner  generated  vs.  instructor  provided  analysis  of 

semantic  relationships.  Paper  Presented  at  the  Annual  Meeting  of  th»  Assnrta^n  frtf  gdiiratinnai 

Communicatk3n&^nd.IechnalogyL  (AECT),  Washington.  D.C.  (Feb.5  -Feb.9, 1992). 
Jonassen.  D.  KL  &  Cole.  P.  (1993).  Learner  generated  vs.  instructor  provided  analysis  of  semantic 

relationships.  Paper  Presented  at  the  Annual  Meeting  of  the  A^nriattnn  fnr  mutational 

rrimmiinfcfltfanfi  and  Technology  (AECT),  NewOrteanS  (Jan.13-17,  1993) 
Keller,  j.  M.  (1983).  "Motivational  design  of  instrucnon".  in  C.  M.  Reigeluth  (Ed.).  Instructional  design 

theories  and  models-  An  overview  of  their  current  status.  Hillsdale,  NJ:  Lawrence  Erilbaum  Associates. 
Kiewra.  K.  A..  &  Frank.  3.  M.  (1988).  Encoding  and  external  storage  effects  of  personal  lecture  notes, 

skeletal  notes,  and  detailed  notes  for  field  independent  and  field  dependent  learners.  Journal  of 

EducationsLBeseaccb.  81(3),  143-148, 
Kenny,  R  F  (1 992).  The  generative  effects  of  instructional  organizers  with  Computer-Based  Interactive 

Video    Paper  Presented  at  the  Annual  Meeting  Of  the  A**ncigtinn  for  Friucatinnal  Communications 

and  Technology  (AECT),  Washington,  D.C.  (Feb.5  -Feb.9, 1992). 
King.  A.  (1991).  Effects  of  training  in  strategic  questioning  on  children's  problem-solving  performance. 

Journatot  EducationaLEsycbotogy,  83(3),  307-317. 
Kozlow.  M.  J..  &  White.  A.  L  (1980).  Advance  organizer  research.  Evaluation  in  Fdnrating  4.  47-48. 
Lasnbtotte.  J  G.  (1993).  Knowledge  maps  for  open-ended  problem  solving.  Paper  Presented  at  the 

Annual  Meeting  Of  American  Fdurationai  Research  AftMriatinr]  /M'ata,  GA  (April,  1993). 
La/nbiotte.  J.  G..  &  Dansereau.  0.  F.  (1992).  Effects  of  knowledge  maps  and  prior  knowledge  on  recall  of 

science  lecture  content.  Journal  of  Experimental  Education,  fi0(3),  189-201. 
Lenze,  J.  S.  (1992).  The  effect  of  varied  instructional  strategies  in  facilitating  subjects'  comprehension  of 

spatial  concepts.  Ufl  polished  Doctoral  Dissertation  The  Pennsylvania  State  University.  1992. 
Lucas.  S.  B.  (1973).  The  effects  of  utilizing  three  types  of  advance  organizers  for  learning  a  biological 

concept  in  seventh  grade  science,  nisserration  Abstract  Internationa  1973.  33.  3390A. 
Luiten.  J  W.  (1S80).  Advance  organi?ers  in  learning  Evaluation  in  Fducation,  4,  49-50. 
Manzo.  A.  V.  (1970).  Improving  reading  comprehensions  through  reciprocal  questioning.  Dissertation 

Abstract  International  30.  5344A-5345A. 
Mayer,  R  E.  (1978).  Advance  organizers:  That  compensate  tor  the  organization  of  test  Journal  of 

PdMoational  psychology ,  70(6).  880-886. 
Mayer.  R.  E.  (1979).  Can  advance  organizers  influence  meaningful  i«flrn»ng7ppyj*w  of  p^atinnai 

afisearcb.  49(2),  371-383. 
Mayer,  R.  E.  (1992).  Cognition  and  instruction:  Their  historic  meetin  within  educational  psychology. 

Journal  Ql  Educational  Psychology.  84(4),  405-41  a 
Mayer,  R  E.  (1989).  Models  tor  understanding.  Review  of  Educational  Research  59(1).  43-64. 
Mayer,  R  E ,  Dyck.  J.  L,  &  Cook.  L  K  (1964).  Techniques  that  help  readers  biuld  mental  models  from 

Scientific  text:  rWinit;nn<  protraintng,  and  signaling  Journal  nf  Frfijratlnnfll  pgytfinlr^y  7ft  1089- 
1105. 

McCagg.  E  C  .  &  Dansereau.  D.  P.  (1991).  A  convergent  paradlg  for  examining  knowledge  mapping  as  a 

learning  strategy.  Journal  of  Educational  Research  84(6),  317-324. 
McLean.  L,  Kai  Yeh,  N.f  &  Reigeluth,  C.  M.  (1983).  The  effects  ot  format  ot  synthesizer  on  conceptual 

learning.  mPAE  working  Paper  No  ia  Syracuse  University.  Syracuse,  NY.  13244-2340. 
Memory.  0.  M.  (1983).  Main  idea  pre-questions  as  adjunct  aids  with  good  and  low  average  middle  grade 

readers.  Journal  of  Pearling  Ftahawnr  15(2),  37-47. 
Newby,  Q.  J.,  A  Stepich,  0.  A.  (1991).  Instructional  analogies  and  the  learning  of  tangible  and  intangible 

concepts.  Paper  Presented  at  the  Annual  Meeting  of  the  Association  tor  PHtir^tmnai  rnmmuniraiinnq 

and  Technology  (AECT).  Orlando,  F!.,  (  February.  1991). 
Norman.  D.  A..  &  Lindsay.  P.  H.  (1977)  Human  intnrmatinn  processing:  An  infraction  to  psychology 

New  York:  Academic  Press. 
Owens.  A.  M.  (1977).  The  effects  ot  question-generation,  question-answering,  and  reading  on  prose 

learning.  Dissertation  Abstract  International  37,  5709A-5701A 

-  •  P  927 


Osborne.  R.  J.,  &  Wittrock,  M.  C.  (1983).  Learning  science:  A  generative  procaas-Sclanca  Education . 
67(4).  489-508. 

Peper.  R.  J.,  &  Mayer,  R.  E.  (1986).  Generative  effects  of  note-taking  dunng  science  lectures.  Journal  ot. 

EducatkanalB^t^togy,  78(1),  34*38. 
Presstey.  G  M.  (1976).  Mental  imagery  helps  eight  years  old  remember  what  they  readJoumal  of 

EducaticmlPsychotogy,  68(3).  355-359. 
Pressley.G.  M.,  Levin.  J.  R..  Kuiper.  N.  A..  Bryant.  S.  L,  S  Mwhoner.  S,  (1982).  Mnemonic  versus 

nonmnemonic  vocabulary  learning  strategies:  additional  comprehension,  ilnumal  nf  Frtiicatinna) 

Psychology,  74(5).  693-707. 
Ragan.  L  C.  (1991).  The  effect  of  a  generative  concept  map  technique  in  a  computer-based  medium  of 

coprehension.  Eapat  Etesaniad  at  ADCIS 1991  Conference. 
Reigeluth.  C.  M.  (Ed.)  (1983).  inaojaional  design  thanrias  and  models;  An  overYiaw  ol  their  suneot  status 

Hillsdale.  NJ:  Lawrence  Ertlbaum  Associates. 
Reigeluth,  C.  M.  (Ed.)  (1987).  Instructional  theories  in  action:  Lessons  illustrating  selected  theories  and 

models.  Hillsdale,  NJ:  Lawrence  Erllbaum  Associates. 
Reigeluth.  C.  M.,  &  Darwazeh,  A.  N.  (1982).  The  elaboration  theory's  procedure  for  designing  instruction: 

A  conceptual  approach.  Journal  ot  Instructional  Development,  5(3).  22-32. 
Reigeluth.  C.  M ,  &  Stein.  F.  S.  (1963).  The  elaboration  theory  of  instruction,  in  C.  M.  Reigeluth  (Ed.), 

InstnJcBQnaLdesiQnJbeoDes  and  models:  An  overview  .of  their  current  status  Hillsdale.  NJ:  Lawrence 

Erllbaum  Associates. 

Rickards,  J-  P  (1976).  Interaction  of  position  and  conceptual  level  of  adjunct  questions  on  immediate  and 

delayed  retention  Of  text.  Journal  Qf  grti  rational  Pc^/rhnfoqY  68(2).  210-217. 

Rickards.  J.  P..  &  August,  G.  J.  (1975).  Generative  underlining  strategies  in  prose  recall  Journal  .of 

Educational  Psychology.  67(6),  660-865. 
Rickards,  J.  P..  &  Hatcher,  C.  W.  (1977-1978).  Interspersed  meaningful  learning  questions  as  semantic 

cues  for  poor  comprehension.  Reading  R**Aamh  ntmrtsiriy  538-553. 
Rigney,  J.  W.  (1978).  Leming  strategies:  A  theoritical  perspective.  In  H.  F.  O'Neal  (Ed.  teaming. 

strategies.  New  York:  Academic  Press. 
Rigney.  J.  W.  (1980).  Cognitive  learning  strategies  and  dualities  in  information  processing.  In  R  E  Snow.  P. 

Frederico.  &  W.  E.  Montague  (Ed )  ftptttudej  earning  and  instruction  vol.  1.  Hillsdale,  NJ:  Lawrence 

Erllbaum  Associates. 

Rigney.  J.  W..  &  Lutz,  K.  A.  (1976).  Effect  of  graphic  analogies  of  concepts  in  chemistry  on  learning  and 

attitude,  .journal  nf  FriiK^nal  Fi^hntogy.  65(3),  305-311. 
Rothkopf.  E.  Z  (1966).  Learning  from  written  instructive  material:  An  exploration  of  the  control  of 

inspection  behavior  by  test-like  events.  ftm^a"  «Hn»atw*i»i  Ri**^"?1  iV^imff  *{A)  241-249. 
Rothkopf.  E.  Z  (1970).  The  concept  of  methemagenic  activates.  Review  nf  Educational  Baaaaicfr  40(3). 

325-336. 

Rothkopf.  EZ..4Bisbicos.E.  E.  (1967).  Selective  facilitative  iff ecti  of  Interspersed  questions  on 

learning  from  written  material.  Journal  nf  Educational  Eaycfttalogy,  58(1),  56-61. 
Sagana,  S.  D..  &  DiVesta,  F.  F.  (1978).  Learning  expectation  Induced  by  adjunct  questions  and  retrieval  ot 

inetentiona)  and  incindental  information,  ilnumal  nf  Educational  Fiynhnlngy,  70(3),  280-286. 
Sanders,  J.  R.  (1973).  Retention  eflects  of  adjunct  questions  in  wrttten  and  aural  discourse.  Journal  of 

Educational  Psychology,  56(2).  181-186. 
ScmkJ,  R.  F..  &  Telaro,  G.  (1990).  Concept  mapping  as  an  Instructional  strategy  for  high  school  biology. 

Journal  of  Educatonal  Research  84(2).  78-85. 
Shavelson.  R.  J.,  Berliner,  D.  C  Ravitch,  M  M.t  &  Leading,  D.  (1974).  Effects  of  position  and  type  of 

question  on  learning  from  prose  material:  Interaction  of  treatments  with  individual  difference.  Journal 

nf  Educattanal  Psynhnhgy.  66(1),  40-48 
Seretny,  M.  L.  &  Dean.  R.  S.  (1986).  Interspersed  post-passage  questions  and  reading  comprehension 

aChievment  ^Intimalnf  EdUCtfOOal  P^hnlngy  78(3),  228-229. 

Shrager,  L.  &  Mayer,  R.  (1989).  Notetaking  fosters  generative  learning  strategies  in  novices.  JoumaLoL 

EducationaLesychcdogyl8i(2).  263-264. 
Simbo.  F.  K.  (1988).  The  effects  of  notetaking  approach  cn  student  achievement  In  secondary  school 

geography.  Journal  nf  Educational  Basaatcb  81(6).  377-381. 
Simmons,  0.  G.  Griffin,  C.  C,  &  Kameenui.  E.  J.  (1988).  Effects  of  teacher  constructed  pre  and  post 

graphic  organizer  instruction  on  sixth  grade  science  students'  comprehension  and  recall.  Journal  at. 


r  1  7 


92ft 


Educational  Rresearch,82(i),  15-21. 
Snapp,  J,  C,  &  Glover.  J.  A.  (1990).  advance  organizers  and  study  tr****™*  tk*  .inum^  nf  Frtuoatinnai 

Research.  83(5),  266-271. 
Spurlin.  J.  E.,  Dansereau.  D  R.  CDonnell,  AM  &  Brocks,  L  W.  (1988).  Text  Pocessing:  Effects  of 

summanzation  frequency  on  text  recall.  The  Journal  of  Experimental  Education,  56(4).  199-202. 
Tuckman.  B.  W.  ( 1 993).  The  coded  eiaborative  outline  as  a  strategy  to  help  students  learn  from  text. 

Pap#r  Presented  at  th#  Annual  Meeting  pf  Affi^iran  Educational  r^a^w^i  association  Atiata,Gn, 

1993. 

west.  C.  K..  Farmer.  J.  A..  &  Wolff.  P.  M.  (ioo9)  ingtn^nat  desiga  \mp\\f*a*nn*  from  cognttfca  science 
MA:  Allyn  and  Bacon. 

Watts.  G.  H.,  &  Anderson.  R.  C.  (1971).  Effects  of  three  types  of  "serted  questions  on  learning  from 

*  prose.  Journal  of  Educational  Psychology,  62(5),  387*394. 
winn.  W..  ZhuU  T.,  &  Schill.  0.  (1991).  Diagrams  as  aids  to  problem  soving:  Their  role  in  facilitating  search 

and  computation.  ETR&D.  39(1).  17-29. 
Winn.  W.  D..  &  Sutherland.  S.  W.  (1989).  Factors  influencing  the  recall  of  elements  in  maps  and  diagrams 

and  the  strategies  used  to  encode  them.  Journal  fit  Educational  Psychology,  81(1).  33-39. ' 

Wtttrock.  M.  C  (1974a).  I  framing      a  g*n»rativf»  prQCCSS  Educational  Penologist  11  (9)  87-95. 

Wittrock.  M  C.  (1974b).  A  generative  model  of  mathematics  education.  Journal  for  Research  \t\ 

Mathematics  Education.  5(4),  181-196. 
Wittrock.  M.  C.  (1985).  Teaching  learners  generative  strategies  for  enhancing  reading  comprehension. 

Theory  into  Practice,  24(2),  123-126. 
Wittrock.  M.  C.  &  Carter.  J.  (1975).  Generative  processing  of  hierarchically  organized  wordsAmfirican 

Journal  of  Psychology.  86(33).  489-501. 

WittrOCk.  M.  C.  (1990).  K*n*raYi\i&  prnr^cAgnf  ftomprfthfrnginn  FHnratinnal  psychologist  ?4(<t),  345- 

376. 

WittrOCk,  M.  C.  (1992).  ^naraitw  loaming  proTA^c**  nf  thft  hrain  Frti  iratinnal  Psychologist  77{A)  531- 
541. 

Wittrock.  M.  C  .  &  Alesandhni.  K.  (1 990).  Generation  of  summaries  and  analogies  and  analytic  and  holistic 

abilities.  American  Educational  Ftesftatch  Journal  27(3),  489-502. 
Wittrock.  M.  C.  (1991).  Generative  teaching  of  comprehension.  The  Elementary  School  Journal,  92(2). 

169-184 

Woodward,  E  L  (1966).  A  comparative  study  of  teaching  strategies  involving  advance  organizers  and  post 
organizers  and  discovery  and  non-discovery  techniques  where  the  instruction  is  mediated  by 
computer.  ( Florida  State  University.  1966).  DissaBaflop  Abstracts  International  1967, 27, 3787A. 


*    .  r  ■ 


929 


High 
aD!  1 1 ties 


jr.- lines 


 Fie'c- 

independent, 


Field- 
Dependents 


Trained 


Un-Trained 


Organized 


Random 


iar 


Unfamiliar 


High 


Medium 


Lew 


n 

CD 

1 

r- 

o 

CD 

o 

G» 

r* 

CD 

0) 

o 

CO 

< 

CO 

-1 


03 
1 


n 

CJ 

"1 

o 

n 

o 

O 

r* 

CD 

CD 

co 

<-» 

•  CO 

CO  o 
cr  <~ 


CD 
CO 


c 

=3 

CD 


—  CD 

cd  a 
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Abstract:  Within  Mathematics  of  Finance  classes  at  the  Smeal  College  of  Business 
Administration  at  Penn  State  University,  lectures  ait  developed  using  Asymetric's 
Toolbook  and  arc  presented  via  a  computer  system.  This  approach  was  implemented 
because  it  has  the  potential  to  effectively  convey  concepts  that  ait  ordinarily  difficult  to 
communicate  to  large  classes  of  students.  For  example,  the  class  can  input  arbitrary 
interest  rates  and  instantly  see  their  effect  on  other  variables. 

In  the  fall  semester  of  1992,  an  analysis  was  conducted  on  a  class  of  58  junior  level 
Actuarial  Science  students.  The  methods  used  to  make  this  analysis  were  to  video  record 
all  class  lectures  with  two  video  cameras.  One  camera  focused  on  the  students  and 
collected  their  physical  reactions  such  as  facial  expressions  and  body  movements  and  the 
second  camera  recorded  images  projected  by  the  computer  system.  Students  also  received 
a  pie  and  post  attitude  survey  to  collect  their  perception  of  the  computer-assisted 
lectures. 


The  Smeal  College  of  Business  Administration  at  the  Penn  State  University  continues  to  investigate 
the  potential  of  computer-assisted  learning  systems.  Within  Mathematics  of  Finance  classes,  lectures  are 
developed  using  Asymetric's  Toolbook  and  are  pr  sen  ted  via  a  computer  system.  This  approach  was 
implemented  because  it  has  the  potential  to  effect:'  ely  convey  concepts  that  are  ordinarily  difficult  to 
communicate  to  large  classes  of  students.  For  example,  arbitrary  interest  rates  can  be  fed  to  the  system  and 
students  can  instantly  see  a  visual  representation  of  their  effect  on  other  variables.  The  instructor  can  ask 
questions  and  input  student  responses  and  electronically  graph  .the  result  during  lectures  to  help  monitor  the 
classes'  auentiveness  or  understanding  of  the  material.  This  approach  adds  additional  dynamics  to  lectures  and 
assignments. 

While  it  seemed,  from  an  instructor  perspective,  that  the  computer-assisted  learning  approach  could 
positively  impact  classes,  it  was  unclear  as  to  how  students  would  react  to  it  A  pilot  study  was  therefore 
conducted  to  collect  student  perceptions  and  reactions  to  this  instructional  approach.  The  objectives  were  to 
make  an  initial  assessment  of  the  computer-assisted  lecture  delivery  format;  to  collect  student  reactions  to 
this  delivery  format;  and  to  investigate  the  potential  of  video  recording  as  a  method  of  data  collection 
because,  among  other  things,  videotaping  has  been  used  extensively  for  subject  observation  with  substantial 
benefits  reported  (Baum  A  Gray,  1992). 
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Methods 


In  the  fall  semester  of  1992,  an  analysis  was  conducted  on  a  class  of  38  junior  level  Actuarial  Science 
students.  In  previous  years,  the  general  characteristics  of  the  classroom  were  as  follows:  each  student  had  a 
copy  of  the  professor's  typed  notes  upon  which  they  wrote  comments;  the  professor  knew  each  student  by 
name;  an  attempt  was  made  to  call  upon  each  student  at  least  once  each  class  to  respond  to  question  related 
material;  and  an  important  part  of  each  lecture  was  explaining  concepts  to  the  "man  on  the  street'*  The 
change  this  semester  was  that  lectures  were  presented  via  a  computer  systems  using  Toolbook. 

There  were  two  methods  of  analysis  used,  One  method  was  to  video  record  all  class  lectures.  For 
video  recording,  the  researchers  used  two  cameras.  One  camera  focused  on  students  to  collect  their  physical 
reactions  such  as  facial  expressions  and  body  movements  and  a  second  camera  recorded  the  visual  images 
projected  by  the  computer  system,  The  researchers  later  combined  the  video  signal  so  that  physical 
reactions  could  be  seen  simultaneously  with  what  was  projected  electronically  by  the  computer.  Thirty-two 
75  minute  class  lectures  were  recorded  of  which  7  are  examined  in  this  paper.  For  analysis,  non-verbal 
communications  as  well  as  classroom  interactions  were  examined  relative  to  the  delivery  format 

The  second  method  used  to  assess  the  impact  of  this  technology  was  an  attitude  survey.  In  week  one 
of  class,  students  were  given  an  introduction  to  computer-assisted  lectures  and  informed  that  throughout  the 
course,  lectures  of  this  type  would  be  given.  At  the  end  of  the  first  week,  students  received  a  pre-attitude 
survey . 

A  post-attitude  survey,  identical  in  nature  to  the  pet -attitude  survey,  was  given  in  the  last  week  of 
class.  The  researchers  compared  surveys  to  observe  if  any  differences  occurred  in  attitude  toward  the  computer- 
assisted  lectures  from  the  beginning  of  the  semester  to  the  end. 

Each  survey  presented  statements  to  which  respondents  rated  their  reactions  on  a  9  point  scale  from  0 
to  3.  The  scale  ranged  from  Strongly  Agree  (0  points),  to  Indifferent  (4  points),  to  Strongly  Disagree  (8 
points).  Of  the  58  students  who  completed  a  pre-attitude  survey,  only  31  completed  a  post-attitude  survey. 
The  analysis  of  survey  data  is,  therefore,  based  on  31  respondents. 

Results  &  Discussion 

Videotaping  retted 

Video  recording  class  lectures  appears  to  have  potential  merit  in  monitoring  student  reactions  to  new 
technological  approaches  in  the  classroom.  The  video  medium  provided  a  permanent  record  of  students  and 
how  they  responded  to  the  computer-assisted  lectures.  Numerous  observations  could  be  made  specifically 
about  the  technology,  as  well  as  the  class  in  general,  and  these  observations  could  be  reviewed  repeatedly  and 
verified  by  others. 

Video  recording  lends  itself  to  the  collection  of  large  amounts  of  various  kinds  of  data,  which  requires 
more  time,  effort,  and  resources  to  analyze.  The  advantages  of  this  technique  relative  to  cost  need  to  be  more 
thoroughly  investigated.  At  present,  however,  the  video  recording  approach  as  presented  in  this  study  appears 
to  be  an  effective  method  for  identifying  student  reactions  to  computer-assisted  lectures. 

Videotaping  results:  classroom  interaction 

For  this  analysis,  tapes  from  five  class  sessions  were  reviewed.  The  researchers  look  at  the  following 
classroom  interactions: 

areas  to  which  the  majority  students  directed  their  attention  (e.g.,  professor's  typed 
notes,  the  professor  or  the  projected  image)  in  response  to  the  computerized  lectures; 
•         areas  to  which  the  majority  students  directed  their  attention  in  responses  to  type  of 
visual  images  projected  (eg.,  graphs). 

Computerized  lectures  can,  in  many  cases,  take  significantly  longer  to  develop  than  traditional 
lectures.  Because  of  this  time  and  labor  intensiveness,  it  is  important  to  ensure  that  the  visual  information 
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projected  effectively  cues  students  and  is  useful  to  (hem.  By  making  observations,  the  researcher  hoped  to 
improve  the  presentation  material  by  identifying  screens  designs  which  caught  student  attention  and  designs 
to  which  students  responded  favorably. 

To  determine  whether  or  not  screen  designs  effectively  cued  student  attention,  the  researchers  counted 
each  time  the  majority  of  students  look  up  or  looked  down  in  response  to  what  was  projected  by  the 
computer.  Four  categories  were  created: 

1 .  No  visual  and  looking  up 

The  number  of  times  the  majority  of  students  were  looking  up  when  no  visual  was  projected 
by  the  computer. 

2.  No  visual  and  looking  down 

The  number  of  times  the  majority  of  students  were  looking  down  when  no  visual  was 
projected  by  the  computer. 

3.  Visual  on  and  looking  up 

The  number  of  times  the  majority  of  students  were  looking  up  when  a  visual  was  projected 
by  the  computer. 

4.  Visual  on  and  looking  down 

The  number  of  times  the  majority  of  students  were  looking  down  when  a  visual  was 
projected  by  the  computer. 

If  the  purpose  of  presenting  visual  information  via  a  computer  system  is  to  promote,  in  some  way, 
student  understanding,  then  it  becomes  important  to  ensure  that  the  visual  effectively  conveys  its  meaning 
and  that  students  attend  to  tL  The  primary  reason,  therefore,  for  identifying  these  categories  was  to 
determine,  in  a  general  sense,  if  students  viewed  the  visual  information  when  it  was  presented.  The  highest 
incidences  occurred  in  the  visual  on  and  looking  down  category  (see  Table  1). 


Table  1 

Response  to  Visuals:  Attention  Areas 


Class  1 

Class  2 

Ctass3 

Class  4 

Class  5 

Total 

No  visual 
St  looking 

up 

13 

0 

12 

1 

3 

31 

No  visual 
St  looking 
down 

10 

0 

7 

13 

0 

30 

Visual  on 
St  looking 

up 

9 

15 

2 

19 

13 

58 

Visual  on 
&  looking 
down 

16 

14 

8 

22 

18 

78 

This  means  the  there  were  78  incidences  when  the  majority  of  students  were  looking  down  for  the  period  of 
time  when  the  visual  was  projected.  It  should  be  pointed  out  that  students  had  the  professor's  typed  notes 
with  which  to  follow  the  lecture.  Thus,  it  is  to  be  expected  that  they  would  page  through  these  notes  while 
the  concept  was  discussed.  Oftentimes  the  professor  talked  through  visuals  using  the  computer  mouse  to 
point  out  key  areas  and  formulas.  Frequently  during  this  time,  a  large  portion  of  the  class  referred  to  the 
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professor's  notes  and  paged  back  and  forth  through  them.  It  appeared  thai,  among  other  things,  the  notes 
enabled  students  to  bring  together  additional  information  that  may  or  may  not  have  been  presented  on  screen. 
The  reliance  on  the  notes  seems  to  provide  an  additional  means  by  which  students  individually  verified  the 
presented  information  and  or  came  to  understand  it 

Given  the  value  of  the  projecting  visual  information  for  the  entire  class  to  view  and  the  way  in  which 
students,  during  class  lectures,  utilized  the  professor's  typed  notes,  it  is  difficult  to  assess  the  effectiveness  of 
screen  designs.  A  future  inquiry  might  include  a  control  group  which  is  presented  (he  visual  information 
unaccompanied  by  the  professor's  notes  to  more  accurately  identify  student  reactions. 

Vidcpaping  results;  reactions  to  visual  types 

The  researchers  wanted  to  identify  the  types  of  visuals  (text,  animated  graphs  or  tables)  whic  h  most 
effectively  cued  student  attention.  It  appeared  that  students  looked  down  or  took  notes  when  visuals 
containing  text  and  formulas  were  presented.  On  most  occasions,  however,  when  the  professor  presented 
graphs  containing  animations,  students  looked  up.  Two  reasons  are  suggested  this  observation.  First,  the 
professor's  typed  notes  contained  formulas  and  replicated  what  was  present  on  screen  and  thus  there  may  have 
been  no  need  for  students  to  look  up.  They  could  easily  follow  the  professor  by  looking  at  his  typed  notes. 
Second,  unlike  text  and  formulas,  animated  tables  and  graphs  can  not  be  duplicated  on  paper.  Animation 
may  have  received  more  attention  because  of  its  distinctiveness.  Animation  as  an  attention  gaining  device 
can  be  effective  since  attention  is  influenced  by  unique  stimuli  Possibly,  animating  tables  and  graphs  was 
unique  enough  to  gain  students1  attention. 

General  observations: 

Video  recording  made  additional  observations  about  the  class  possible  and  some  of  these  observations 
are  presented  below. 

For  five  of  the  class  sessions,  the  number  of  student  initiated  interactions  with  the  professor 
were  recorded  Each  time  a  student  commented  or  asked  a  question,  it  was  considered  an 
interaction.  If  a  student,  for  example,  asked  a  question  and  then  followed  up  immediately 
with  a  second  question,  then  it  was  counted  as  one  interaction.  There  were  83  occurrences 
when  students  initiated  interaction  with  the  professor*  32  of  which  were  initiated  by  males 
and  51  by  females.  Such  information  may  prove  useful  to  the  professor  since  classroom 
interaction  among  all  students  was  highly  encouraged. 

Cueing  students  with  key  words  is  attention  getting.  Students*  as  observed  in  this  study, 
spent  much  time  following  along  with  paper  handouts  and  often  ignored  what  was  projected 
on  screen.  For  example,  when  the  professor  said.  1  have  a  picture  here."  students  looked  up, 
and  on  subsequent  occasions  the  word  "picture"  appeared  to  get  their  attention. 

Much  can  be  gained  from  observing  students  and  watching  their  body  language  and  reactions. 
In  some  cases  an  observer  can  readily  tell  when  students  do  not  understand  or  are  confused 
(Alessi  &  Trollip,  1985).  Throughout  the  course  the  professor  constantly  challenged 
students  by  posing  questions.  Students  often  displayed  behaviors  like  smiling  or  looking 
downward  when  they  did  not  know  the  answer.  It  was  also  dear  by  viewing  the  video  tapes 
when  more  than  one  student  was  having  difficultly  with  a  concept  or  question.  In  such  cases 
students  looked  to  one  another,  shook  their  head  or  made  a  facial  expression. 

Attitude  Survey  Results 

On  a  survey,  subjects  ranked  the  number  of  computer  courses  they  had  completed  on  a  10-point  scale 
ranging  from  0  to  9  or  more  completed  courses.  Completed  computer  courses  was  used  as  a  measure  of 
computer  knowledge  (Rattanapian,  1992).  The  researcher  obtained  this  measure  to  determine  if  knowledge 
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about  computers  influenced  perceptions  of  (he  lectures.  On  average,  students  completed  2  computer  courses. 
Figure  1  shows  the  percentages  of  respondents  by  number  of  courses  completed. 

Courses  Completed 


0        20       40       60       80  100 

%  Responding 
Figure  L  Computer  Courses  Completed 


Given  the  proliferation  of  technology  in  today's  classrooms  and  the  use  of  computers  to  present 
information  electronically,  the  researchers  sought  to  identify  the  extent  to  which  students  in  this  class  had 
been  exposed  to  computerized  lectures.  On  the  pie-survey,  students  ranked  their  experience  and  on  average 
reported  having  no  experiences  with  the  lecture  format  (see  figure  2). 

Experience  Level 
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Figure  2.  Experience  with  Computer-Assisted  Lectures 

Despite  the  lack  of  exposure,  students  anticipated  that  the  computer  would  not  make  lectures  more 
complex  (see  figure  3).  On  the  survey,  for  example,  students  were  asked  if  they  thought  the  computerized 
lectures  would  be  easy  to  follow.  Responses  to  this  item  were  positive  and  this  perception  remained 
constant  from  the  pre  to  the  post  survey. 
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Figure  3.  Lectures  were  easy  to  follow  pre  and  post  surveys 


S  tudenis  anticipated  that  the  lectures  would  be  informative  (see  figure  4).  On  the  survey,  students 
were  presented  the  following  item,  "Computer-assisted  lectures  will  be  informative."  Responses  tended  to  be 
on  the  positive  side  and  (his  perception  remained  constant  throughout  the  semester. 


Sfc  Bctpandigs 

50 


Figure  4.  Lectures  were  informative  pre  and  post  surveys 


Computer  familiarity  and  attitude 

Classroom  computers  are  neutral  in  design  but  lack  neutrality  often  times  in  the  way  they  are  used 
Jones.  1987)  and  perceived.  Males,  for  example,  have  been  found  to  readily  express  interest  in  computers 
while  females  are  less  likely  to  do  so  (Miura,  1987).  Students  in  this  study  had  varying  levels  of  computer 
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expertise  ranging  from  novice  to  experienced  users.  The  researchers  were  interested  in  determining  if 
computer  familiarity  effected  (positively  or  negatively)  students'  attitude  toward  lectures.  As  a  measure  of 
familiarity,  number  of  computer  courses  completed  was  used  based  on  the  assumption  that  some  familiarity 
with  computers  is  required  with  course  completed.  Familiarity  did  not  appear  to  negatively  bias  students' 
opinion  of  lectures  and  in  fact,  students  appeared  to  look  forward  to  the  teaching  format 

At  the  beginning  of  the  semester,  in  the  pre-attitudc  survey,  students  were  asked  if  they  thought  the 
lectures  would  dramatically  impact  the  way  in  which  the  course  was  taught  Responses,  examined  in  terms 
of  computer  familiarity,  show  that  for  some  respondents  the  perceived  degree  of  impact  of  the  computerized 
lectures  decreased  as  the  number  of  computer  courses  completed  increased  (see  figure  5).  Thus,  those  having 
completed  more  computer  courses  felt  that  the  computerized  lectures  would  have  less  of  an  impact  than  those 
with  fewer  courses.  This  may  suggest  that  those  with  more  courses  completed  understand  the  technology  and 
its  limitations  to  a  greater  extent  than  those  with  fewer  courses.  It  should    pointed  out,  however,  that  only 
7  students  completed  more  than  3  courses  and  the  majority  of  respondents  completed  fewer  than  2  courses. 
From  such  a  small  sample,  it  is  difficult  to  assess  whether  or  not  this  trend  would  persist  for  larger 
populations. 

Number  of  Courses 
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Average  Rating  by  Category 

Figure  5.  Perceived  Impact  by  Number  of  Computer  Courses 

On  the  pre -attitude  survey,  students  indicated  whether  or  not  they  would  feel  comfortable  being  taught 
with  computerized  lectures.  For  this  analysis,  there  were  15  out  of  the  32  respondents  who  had  completed 
fewer  than  3  computer  courses  and  they  indicated  that  they  would  feel  comfortable  being  taught  with 
computerized  lectures  (see  figure  6).  Of  these  15  respondents,  fewer  computer  courses  completed  coincided 
with  favorable  ratings.  There  appeared  to  be  a  tendency  for  those  who  had  completed  fewer  courses  to 
indicate  that  they  would  feel  comfortable  being  taught  with  computerized  lectures.  Again,  because  of  the 
lack  of  distribution  of  students  across  the  different  levels  of  courses  completed,  it  should  not  be  concluded 
that  those  with  more  familiarity  would  feel  uncomfortable  being  taught  with  computerized  lectures.  On  the 
post-survey,  attitudes  toward  being  comfortable  with  computerized  lectures  seemed  to  even  out 
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Figure  6.  Comfort  by  Computer  Courses  completed:  pre  survey 

Animations  and  attitude 

Many  of  the  lectures  included  animations  of  tables  and  graphs  to  visually  represent  concepts  and  ideas. 
The  professor  could,  for  example,  when  discussing  interest  rates,  ask  students  to  suggest  an  appropriate  rate 
of  interest  The  given  rate  could  be  input  into  the  computer  and  by  animating  the  graph  or  table  effects  on 
other  variables  could  be  instantly  represented.  Students  perceived  the  animation  of  concepts  positively  both 
prior  to  their  exposure  to  animation  and  after  long-term  exposure  to  it  From  the  data  collected  students 
anticipated  at  the  beginning  of  the  semester  that  animations  of  graphs  would  be  attention  getting  and  useful 
(see  figure  7).  This  perception  remained  constant  for  the  most  part  from  the  beginning  of  the  semester  to 
the  end 
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Figure  7.  Animating  Graphs:  pre  and  post  survey 
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Summary 


This  pilot  study  attempted  to  make  an  initial  assessment  of  student  reactions  to  computer-assisted 
lectures  and  thus  no  efforts  was  made  to  control  for  extraneous  factors  which  may  have  influenced  student 
attitudes  during  the  course  of  the  semester.  Given  the  constraints  of  this  study,  the  researchers  hesitate  to 
make  any  generaiizable  conclusions,  however,  some  interesting  observations  were  made.  First,  video 
recording  class  lectures  with  two  video  cameras  is  an  effective  technique  to  assess  the  implementation  of  a 
new  classroom  technology.  Video  preserves  rich  details  about  observations  which  can  be  studied  repeatedly 
and  validated  by  other  researchers  (Van  Dalen,  1979).  One  drawback  to  this  technique  is  that  video  molding 
lends  itself  to  the  collection  of  large  amounts  of  data  and  consequently  requires  substantial  resources  in  time, 
money  and  energy  to  make  thorough  analyses.  Secood,  upon  review  of  the  video  recordings  it  appeared  that 
students  tended  not  to  look  at  visuals  which  included  text  or  formulas  and  appeared  more  attentive  to 
animated  tables  and  graphs.  This  was  expected  because  studen's  had  the  professor's  typed  notes  with  which 
to  follow  the  lectures.  Third,  the  subject  population  had  very  little  computer  knowledge  or  exposure  to 
computer-assisted  lectures.  Results,  however,  from  the  prc-survey  show  that  students  anticipated  that  the 
lectures  would  be  easy  to  follow  and  informative  and  that  they  would  feel  comfortable  being  taught  with  this 
delivery  format  The  researchers  were  concerned  that  students,  without  much  computer  familiarity,  might  be 
skeptical  about  the  delivery  format  which  may  impact  learning.  Fourth,  students  appeared  to  perceive 
animation  of  concepts  and  ideas  as  useful  and  attention  gaining.  Moreover,  its  use  in  this  study  whereby  the 
professor  maintained  an  interactive  dialogue  with  students  and  fed  their  responses  into  the  computer  and 
animated  results,  from  the  researchers'  perspective,  added  additional  dynamics  to  class  lectures  and 
assignments.  This  technique  also  helped  to  monitor  student  attentiveness  in  a  large  lecture  room. 
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ABSTRACT 

This  ethnographic  study  examines  the  process  of  implementing  an 
interactive  videodisc  program  in  a  senior  high  school.     The  research 
questions  were:  What  happened  during  the  implementation  process  and  what 
factors  facilitated  or  hindered  the  implementation  of  the  innovation? 
Descriptive  data  were  collected  through  participant  observation, 
interviews,  and  artifact  collections.     Study  results  revealed  that 
implementing  an  educational  innovation  is  a  complex,  multidimensional 
process  invoking  the  organization,  the  people,  and  the  innovation. 
Effective  imp^gentation  depends  on  the  combination  of  all  the  factors 
described  in  tne  study,  including  initiation;  planning,  preparation,  and 
support;  communication  and  collaboration;  district  and  school  policies; 
teachers'  realities;  students k  characteristics;  change  strategies;  and 
characteristics  of  the  innovation. 
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lat reduce ion 

As  technology  development  continues,  educational  institutions  are 
planning  and  expanding  technology-related  learning  experiences.  However, 
the  implementation  on  a  technology-related  educational  innovation  is  a 
highly  complex  process  involving  relationships  between  end-users  and 
change  agents;  people  and  technology.     The  implementation  often  involves 
changes  in  curriculum,  changes  in  instructional  approaches,  changes  in 
people's  attitudes,  behaviors,  knowledge,  skills,  and  changes  in 
interpersonal  interactions  among  people.    Much  of  the  research  on 
educational  innovations  is  characterized  by  a  heavy  reliance  on  recall 
from  memory,  a  short  time  frame,  and  after-process  data  gathering 
(Rogers,  1983).    Such  research  tends  to  neglect  the  in-process 
complexities  and  difficulties  of  implementation  which  have  a  vital 
influence  on  the  eventual  adoption  of  an  educational  innovation.  Rogers 
further  suggested  that  innovation -related  research  methodologies  should 
include  longitudinal  investigations,  broader  contexts,  multiple 
perspectives,  and  in-process  studies.     Hord,  Rutherford,  Huling-Austin, 
and  Hall  (1987)  also  noted  that  efforts  to  understand  the  change  process 
should  concurrently  focus  on  the  individuals,  the  innovation,  and  the 
context  of  the  innovation.    The  research  reported  herein  followed  the 
above  guidelines  when  examining  the  implementation  of  an  educational 
technology  innovation  in  a  high  school. 

Purpose  and  Questions  of  the  Study 

The  purpose  of  chis  study  was  to  investigate  and  analyze  the 
process  of  implementing  an  interactive  videodisc  program  in  a  high 
school.     The  design  of  the  study  was  an  ethnographic  case  study  which 
involved  intensive  data  collection  on  many  instances  of  the  phenomenon 
over  an  extended  time  period.    The  study  attempted  to  reflect  the 
various  dimensions  of  the  implementation  process  from  the  point  of  view 
on  the  people  involved,  primarily  the  teachers,  the  media  specialist, 
and  the  administrators.     The  research  questions  of  the  study  were: 

1.  What  happened  during  the  implementation  process? 

2.  What  were  the  factors  that  facilitated  or  hindered  the  implementation 
of  the  innovation? 

Methodology 

Rationale  for  Ethnographic  Case  Study  Approach 

The  basic  assumption  underlying  the  research  design  was  that 
implementation  of  an  educational  innovation  is  a  change  process 
occurring  over  time  and  involving  th©  interaction  of  the  people,  the 
innovation,  and  the  context.    All  thosa  three  elements  were  considered 
simultaneously  in  this  study  by  using  an  ethnographic  approach,  thereby 
creating  a  completa  and  credible  study  of  the  implementation. 
Setting 

A  mid-west  comprehensive  high  school,  referred  to  by  the  pseudonym 
-North  High  School  (NHS),-  was  selected  as  the  setting  for  this  study. 
Selection  of  this  school  was  made  because  the  school  was  chosen  by  the 
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Library  of  Congress  as  che  cast  sice  for  an  interactive  videodisc 

program  --  American  Memory. 

Innovation 

The  educational  innovation  that  was  examined  in  this  study  was  the 
American  Memory  program.    The  program,  collections  of  historical  primary 
sources,  included  photographs,  manuscripts,  motion  pictures,  books,  and 
sound  recordings  appropriate  for  use  in  American  History  classes.  The 
hardware  of  the  program  included  a  Macintosh  microcomputer,  a  CD-ROM 
player,  a  videodisc  player,  and  a  television  monitor. 
Participants 

The  selecting  and  sampling  pro  «:-.s  in  this  study  included 
strategies  for  expanding  the  scope  ^    the  study,  refining  the  questions 
or  constructs  under  investigation,  and  generating  new  lines  of  inquiry. 
In  general,  the  study  employed  two  means  of  participant  selections: 
criterion-based  or  "purposive  sampling"  (Patton,  1980)  and  "snowballing* 
(Delaxnont,  1992).    The  participants  of  the  study  included:  program  users 
(8  teachers),  program  coordinator  (1  media  specialist),  program 
initiators  (2  district  administrators       media  and  social  studies 
coordinators),  school  administrators  and  staff,  and  students. 
Data  Collection  Methods 

The  ethnographic  approach  used  multiple  data  collection  strategies 
to  provide  the  flexibility  needed  to  study  diverse  aspects  of  setting, 
innovation,  and  participants .    Participant  observation,  formal 
interviews,  informal  conversations,  and  artifact  collection  were  the 
primary  data  collection  methods  of  this  study.    The  combination  of 
multiple  data  collection  strategies  provided  more  complete  and  complex 
data  than  do  uniiaodal  research  designs,  therefore  enhancing  the 
credibility  of  study. 
Analysis  and  Interpretation  Strategies 

A  "concurrent  approach*  (Marshall  &  Rossman,  1989)  was  employed  in 
which  data  analysis  is  simultaneous  with  data  collection.    On  the  basis 
of  the  strategies  suggested  by  Goer.z  and  LeCompta  (1984)  ,  the  data 
analysis  included  theorizing,  sequential  selection,  and  general  analytic 
procedures.    The  researcher  looked  for  "repeatable  regularities"  of  the 
phenomena  and  events  (Miles  &  Huberman,  1984)  .    Themes  and  subthemes 
representing  the  recurring  patterns  were  developed  to  indicate  the  ebb 
and  flow  of  the  implementation  process.    Together  the  themes  and 
subthemes  answered  the  research  questions;  they  indicated  the 
interrelatedness  of  the  factors  involved  with  the  setting,  the 
innovation,  and  the  people  during  the  implementation  process. 

Findings 

After  examining  the  implementation  process,  the  data  were  analyzed, 
categorized,  and  conceptualized  into  the  major  themes,  which  together 
tell  a  complete  story  of  the  implementation  phenomenon  and  reveal 
dynamic  interrelation  of  the  setting,  the  innovation,  and  the  people 
which  influenced  the  implementation  processes.    The  themes  and  subthemes 
take  their  form  from  recurring  patterns  which  have  similar 
characteristics.    The  five  themes  are:  agendas  and  actions;  teachers' 
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rsalicios;  the  haves  and  the  have-nots;  the  role  of  the  internal  program 
coordinator;  and  the  reactions  to  the  innovation. 
Agendas  and  Actions 

In  general,  the  American  Memory  program  was  initiated  by  the 
district  administrators  without  bottom-up  participation  of  the  teachers 
and  the  media  specialist  at  the  school  Isvei.     In  fact,  the  initiation 
was  mainly  opportunism  —  the  program  was  there,  and  the  grant  was  net 
difficult  to  'write  for.     Minimal  instructional  needs  or  curricular  goals 
were  identifies. 

The  initiation  was  not  followed  by  rarefui  planning  and  preparation, 
by  clarified  staffing  and  responsibilities,  by  effective  communication, 
by  proactive  change  strategies,  or  by  the  provision  of  sufficient 
resources . 

It  was  also  worth  noting  that  Che  district  administrator,  the  media 
specialist,  and  the  teachers  were  making  efforts  to  enhance  the 
technology  inventory  of  the  school.     However,  no  relationships  were 
identified  or  made  between  the  technology  equipment  and  the  school's 
instructional  needs  or  curricular  goals. 
Teachers  1  Realities 

The  teachers'  working  conditions  were  time-bounded  and  management  - 
conscious.    The  teachers  had  to  attend  to  various  responsibilities  such 
as  classroom  management,  material  coverage,  administrative  duties, 
professional  development  demands,  and  school  and  district  policies. 
Surrounded  by  multiple  realities,   teachers  were  faced  with  the 
difficulty  of  "squeezing"  the  American  Memory  program  into  their 
practice  or  discussing  the  program  with  their  colleagues. 

Additionally,  competing  with  other  instructional  demands  required 
by  the  school  district,  the  American  Memory  program  was  not  considered 
as  a  top  priority  by  the  teachers.    While  exercising  their  limited 
autonomy ,  the  teachers  were  likely  to  rule  out  using  the  American  Memory 
program  in  their  classroom  instruction. 
The  Haves  and  the  Have-Nots 

Like  a  cafeteria  providing  differer.c  foods  for  different  customers, 
the  NHS  offered  different  courses  for  the  different  students.  Also, 
like  a  cafeteria  designing  foods  for  different  classifications  of 
customers,  the  school  designed  courses  as  an  intellectual  diet  for 
different  students1  categories.     Not  every  student  could  take  any  course 
he  cr  she  .;anted.     In  order  to  cake  certain  courses,  students  had  to 
meet  certain  requirement  . 

The  tracking  and  ability  grouping  in  NHS  had  a  great  impact  on  the 
students  in  terms  of  the  distribution  of  learning  opportunit iea ,  day-to- 
day school  experiences,  classroom  climates,  utilization  of  educational 
resources,  and  access  to  the  American  Memory  program.     In  addition,  the 
teachers'  expectations  and  instructional  strategies  also  differed  for 
the  various  ability. 
Role  of  Internal  Program  Coordinator 

Using  the  school  media  specialist  as  an  internal  program 
coordinator  resulted  in  mixed  outcomes.     The  familiarity  of  the  school 
environment  and  the  technological  skills  the  media  specialist  already 
had  were  very  important  to  utilizirg  the  program.     However,  the  multiple 
working  loads  of  the  media  specialist  seemed  to  have  negative  impact  on 
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the  effectiveness  of  the  program  implementation.     The  roles  of  Che 
schccl  Tied i. a  specialist:  can  be  divided  into  formal  and  informal 
categories.     The  formal  and  informal  roles  constructed  media 
specialist's  work  span* 

In  addition,  the  distribution  of  authority  in  the  school  seemed  not 
to  equip  the  media  specialist  with  enough  power  to  help  her  m 
implementing  the  program. 
Reactions  to  the  Innovation 

Throughout  the  implementation,  the  American  Memory  program  received 
Limited  use  because  it  was  not  effectively  integrated  into  the 
instruction  and  because  the  teachers  and  students  had  limited  taowledge 
about  the  technology  and  the  program.     Besides,  the  value  of  'primary 
sources-  was  seldom  mentioned.    The  curricular  goal  and  instructional 
objective  of  integrating  the  American  Memory  program  into  the  senior 
high  school  social  studies  curriculum  were  not  identified  or  clarified. 

In  addition,  it  was  unclear  if  the  use  of  primary  sources  was  a 
goal  for  social  studies  instruction  or  just  a  novelty  used  for 
entertainment,  attention-getting,  or  enhancing  assignments  with  visuals. 
It  was  also  questionable  whether  using  primary  source  material  in  senior 
high  school  social  studies  curriculum  will  even  establish  it 
significance.     in  stead  of  exploring  and  enjoying  the  interactive 
learning  experience  as  proposed  by  the  Library  of  congress,  district 
administrators ,  and  media  specialist,  the  teachers  and  students 
sometimes  felt  frustrated  while  using  the  program. 

Implications 

Implementation  of  an  educational  innovation  is  a  complex, 
multidimensional  process.    The  study  was  not  able  to  identify  any 
discrete  factor  as  dominating  the  implementation  process.    Rather,  all 
the  themes  and  subthemes  presented  in  the  study  should  be  taken  together 
to  represent  the  critical  factors  in  the  implementation  process.  Seven 
implications  are  drawn  for  understanding  the  process  of  implementing  the 
interactive  videodisc  program: 

Implication:  The  top-down  initiation  of  an  innovation  with  little 
bottom-up  participation  may  impede  the  implementation  of  the  innovation. 

The  findings  of  the  study  indicated  that  the  school  district 
administrators  initiated  the  innovation  with  very  little  involvement  of 
teachers  or  the  media  specialist.    The  teachers  and  the  media  specialist 
perceived  the  program  as  just  one  of  the  various  demands  put  upon  them. 
This  top-down  initiation  with  little  bottom-up  participation  seemed  to 
result  in  the  lack  of  teachers1  commitment  to  or  interest  in  the 
implementation.     Such  findings  re-emphasized  a  finding  in  the  literature 
The  less  teachers  are  involved  in  the  decision  making  processes,  the 
lower  their  acceptance  and  support  for  the  innovation  and  the  lower 
their  morale  regarding  adoption  of  the  innovation  (Combs,  199 L;  Louis  fit 
Miles,  1990) . 

However,  Salisbury  (1992)  indicated  that  in  order  to  have  large- 
scale  change  it  is  very  iupurtant  to  have  proper  sponsorship  from  the 
tcp  so  the  innovation  will  be  sustained.    Salisbury  argued  that  the 
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bottom-up  modei  can  only  facilitate  small-scale  change,  because  an 
innovation  usually  requires  support  teycnd  che  resources  of  an 
individual  school.     People  at  the  top  are  the  ones  who  can  provide 
pressure  as  well  as  support,  such  as  equipment,   funding,  personnel,  and 
rewards.     In  this  study,  if  people  at  the  top  had  provided 
implementation  pressure  and  support,  the  program  might  have  been  more 
fully  implemented. 

Marsh  (1983)  suggested  that  the  blending  of  top-down  initiative  and 
bottom-up  participation  is  often  a  characteristic  of  successful 
implementation.    The  combination  of  pressure  and  support  from 
educational  leaders  along  with  high  levels  of  participation  from  school 
personnel  is  probably  the  more  effective  approach. 

Implication:  The  implementation  planning,  teacher  preparation,  and  on- 
going monitoring  mav  affect  the  implementation  process. 

The  study  showed  that  there  was  a  lack  of  planning  for  implementing 
the  program  to  fit  into  the  educational  system.    The  curricular  goals 
and  instructional  objectives  were  not  identified.     Ambiguous  strategies 
and  unspecified  work  division  preceded  anxiety,  uncertainty,  and 
confusion  for  the  people  involved.    As  many  research  findings  indicated, 
an  innovative  program  imported  from  cutsiae  an  organization  not  only 
needs  to  be  modified  to  improve  the  "fit*  between  the  innovation  and  che 
conditions  in  the  system;  but  the  structure  of  the  organization  may  also 
have  to  be  changed  to  accommodate  the  innovation;  otherwise  the  ability 
of  people  to  deal  with  the  innovation  will  be  impeded  (Fullan,  1991; 
Guskey,  1988;  Louis      Miles,  L990;  Rogers,  1983),     In  this  case,   if  the 
program  was  taken  as  a  priority  of  the  school  district  and  teachers  were 
provided  with  time  and  incentives,  the  program  might  have  had  a  better 
chance  to  be  more  fully  implemented. 

The  findings  also  indicated  that  the  district  administrators' 
preconception  of  innovation  adoption  was  somewhat  simplistic.  Their 
reliance  on  a  single  inservice  program  to  prepare  teachers  was  neither 
realistic  nor  appropriate,  because  each  teacher  had  a  different  reaction 
toward  the  program.     In  addition,  teachers  said  they  needed  to  know  more 
about  the  program  in  order  co  be  able  to  use  it  effectively.    The  need 
to  respond  to  teachers1   individual  concerns  and  the  need  for  additional 
teacher  training  are  obvious.    The  findings  were  consistent  with  Che 
argument  of  Hord,  Rutherford,  Huling-Austin,  and  Hall   (1987)  that  one  of 
the  most  common  and  serious  mistakes  made  by  administrators  and  leaders 
in  the  change  process  is  to  presume  that  once  a  program  had  been 
introduced  and  initial  training  has  been  completed,  the  users  will  put 
the  program  into  practice.     A  serious  mistake  is  to  assume  that  all 
users  of  the  program  will  react  in  a  similar  way. 

Teachers  in  the  study  indicated  that  the  pre- implementation 
training  was  insufficient.    One-shot  workshop  prior  to  and  even  during 
implementation  is  found  to  be  not  very  helpful.    As  Wedman  a 98 6) 
suggested,  an  inservice  program  introducing  an  entire  package  of 
computer  technologies  can  only  serve  to  raise  teachers1  awareness  an  the 
beginning  stage  of  the  implementation.     Individuals'  concerns  are  often 
focused  on  different  parts  of  the  technology.     During  implementation, 
teachers  need  to  be  offered  additional  training  on  skills  and  knowledge 
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which  deal  with  the  computer ,  HyperCard,   reference  search  techniques, 
and  the  videodisc  player.    Additionally,  and  perhaps  mere  importantly, 
teachers  need  to  be  provided  with  time  to  learn  and  plan  curricuiar 
integration  of  the  program. 

Although  the  innovation  under  study  was  initiated  by  upper  level 
educators,  the  process  of  implementation  was  not  monitored.     In  fact, 
people  involved  paid  only  minimal  attention  to  their  obligations.  The 
attitude  of  "let's  play  with  it  and  see  what  happens"  seemed  to  be 
prevalent  in  the  organization.     The  belief  that  'nothing  will  be  perfect 
for  the  first  year, ■  even  though  the  program  was  proposed  by  the 
administrators  to  fester  an  atmosphere  of  risk-taking,  seemed  to  serve 
as  an  excuse  for  people  to  pull  back  rather  than  move  forward.  Fullan 
(1991)  noted  that  monitoring  the  innovation  process  serves  two  functions. 
First,  by  making  information  on  the  program  available,  it  provides 
access  to  good  ideas.    Many  good  practices  go  unreported  because 
teachers,  schools,  and  districts  are  isolated  from  each  other.  Second, 
monitoring  exposes  new  ideas  to  scrutiny,  helping  to  weed  out  mistakes 
and  further  develop  promising  practices.     Neither  of  these  advantages 
were  realized  at  NHS. 

Implication:  Implementation  without  frequent  communication  among  the 
people  involved  mav  impede  the  process  of  innovation  implementation. 

The  study  revealed  that  the  communication  network  did  not  run  very 
effectively.    Specifically,  there  was  little  communication  and 
discussion  among  the  initiators,  program  coordinator,  and  users.  Two 
reasons  may  apply:  there  was  no  information  system;  and  people  were  too 
busy  with  other  duties. 

The  lack  of  an  established  information  system  at  NHS  was  evident  in 
two  ways.     First,  the  newsletters  were  not  circulated  among  the 
administrators  and  the  teachers.     Second,  data  was  not  gathered  on  how 
the  program  was  being  used  in  the  classrooms  and  school.     As  a  result, 
the  district  administrators  were  unaware  of  the  progress  of  the  program. 
Ornstein  &  Hunkins  (1988)  cautioned  that  communication  channels  during 
the  implementation  process  must  be  kept  open,  and  the  communication 
network  must  be  comprehensive,  so  that  the  people  involved  can  be 
allowed  to  input,  discuss,  and  interact  with  each  other. 

The  findings  also  revealed  that  teachers  did  not  have  time  to  learn 
from  other  colleagues.     Partly  because  of  the  physical  isolation  and 
partly  because  of  the  norms  of  not  observing  or  discussing  each  other's 
work,  teachers  did  not  develop  a  common  "technical  culture"  (Fullan, 
1991,  p.  119).     Therefore,  teachers  did  not  have  clear  and  exact 
information  about  the  program;  the  knowledge  of  the  program  was  not 
diffused  among  the  teachers.    The  finding  was  consistent  with  the 
research  evidence  that  teachers  interact  with  each  other  infrequently 
(Goodlad,  1984;  Rosenholtz,  1989).     Sarason  (1990)  commented  that  the 
isolation  of  teachers  in  the  school  organization  negatively  Impacts  on 
the  innovation  implementation. 
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Implication:   Implementation  with  little  understanding  of  teachers ' 
realities  mav  impede  the  use  of  an  innovation. 

The  study  showed  that  teachers '  realities  were  defined  by  diverse 
influences.     Teaching  at  NHS  was  always  bound  and  influenced  by  the 
policy  and  opinions  from  the  state,  district,  community,  and  parents. 
Teachers'  realities  were  also  shaped  by  school  policy,  schedules, 
activities,  curriculum  coverage  responsibilities,  students' 
characteristics,  and  teachers'  personal  knowledge  and  skills. 

The  teachers  in  this  study  faced  the  program  with  various  reactions  . 
The  reactions  were  influenced  by  both  the  organizational  and  personal 
dimensions  of  teaching.    Organizational  dimensions  included  an  emphasis 
on  academic  achievement,  a  focus  on  subject,  and  a  perception  among 
teachers  that  they  had  little  influence  on  policy.     Personal  dimensions 
included  level  of  experiene^  with  technology  and  length  of  service  with 
Che  school  district. 

The  school  culture,  with  its  strong  emphasis  on  students1  academic 
achievement,  did  not  give  teachers  much  cause  for  implementing  a  program 
which  was  not  perceived  as  relevant  to  academic  achievement.  The 
teachers  mentioned  the  competing  demands  of  several  teaching  improvement 
programs  promoted  by  the  district  which  often  cause  confusion  and 
anxiety  for  the  teachers.    Rosenholtz  (1989)  cautioned  that  an 
innovation  given  less  emphasis  by  school  and  district  administrators 
will  have  less  of  a  chance  to  be  accepted  by  the  teachers.  Since 
American  Memory  program  was  not  proposed  as  the  top  priority  of  the 
school  district,  it  was  less  likely  to  be  implemented  by  teachers. 

The  findings  of  the  study  indicated  that  teachers  who  frequently 
used  computers  and  media  services  showed  stronger  intentions  to  use  the 
program.     For  those  who  were  not  comfortable  with  the  new  technology, 
the  resistance  to  the  program  tended  to  be  much  greater.    Such  tendency 
is  consistent  with  the  finding  indicated  in  the  literature  that  teachers 
who  experience  satisfaction  in  using  a  technology  are  more  accepting  of 
Che  computer  related  innovation.    And  teachers  who  express  high'levels 
of  enthusiasm  toward  computers  are  more  likely  to  be  heavy  computer 
users  'Schofield  &  Verban,  1983). 

Implication:  The  grouping  of  students  by  ability  may  affect  toachers' 
expectations  and  students1  access  to  educational  technology. 

The  study  findings  indicated  that  teachers'  expectations  toward 
students  affected  their  selection  of  teaching  strategies  and  resources 
for  the  students.    Teachers'  expectations  toward  different  ability 
groups  of  students  seemed  to  result  in  differential  treatments  of 
students.     Although  this  study  did  not  attempt  to  answer  questions 
dealing  with  the  relationship  between  teachers'  expectations  and 
students'   learning,  the  findings  of  the  study  seemed  to  be  consistent 
with  past  research  on  teachers'  expectations. 

Research  evidence  is  available  to  demonstrate  that  some  teachers 
provide  different  ability  students  with  sharply  different  learning 
opportunities,  with  students  of  high  potential  tending  to  receive  more 
stimulating  environments  (Good,  1987;  Oakes,  1985,   1992;  Oakes  &  Lipton, 
1990)  .     This  rich-get-richer  scenario  occurred  in  NHS  classrooms.  The 
interactive  videodisc  program  was  perceived  as  complicated  and  novel  by 
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cha  ' eachers .    Therefore,  the  teachers  made  the  decision  not  to  have 
"lass  competent*  low-tracked  students  the  opportunity  to  use  the  program. 
Such  a  tendency  is  consistent  with  the  evidence  of  class-,  race-,  and 
ability-based  differences  in  computer  use  'Apple,   1992;  Sutton,   1991) , 
raising  issues  of  equality  and  equity  cf  access  to  educational  resources 
by  different  ability  student  groups. 

Implication:  The  change  agent  responsibilities  and  strategies  may  affect 
the  implementation  of  an  educational  innovation. 

Using  a  media  specialist  as  an  internal  change  agent  in  the  school 
setting  seemed  to  result  in  mixed  effects.    One  of  the  advantages  was 
that  the  media  specialist  was  knowledgeable  about  the  interactive 
videodisc  technology  and  had  credibility  in  terms  of  providing  technical 
assistance  to  the  teachers.     In  addition,  such  an  in-house  expert  was 
more  familiar  with  the  organization  and  teachers.     However,  questions 
arose  regarding  the  overloaded  responsibilities  of  the  media  specialist 
and  the  cordial  relationship  she  tried  to  maintain  with  the  teachers, 
with  too  many  roles  --  formal  and  informal  —  to  fulfill  and  too  little 
time  to  undertake  them,  the  media  specialist  could  not  fully  attend  to 
the  implementation  of  the  program.     In  addition,  the  media  specialist, 
in  fact,  had  no  power  over  the  teachers.     As  a  result,  there  was  hardly 
any  pressure  which  is  necessary  for  successful  implementation. 

The  media  specialist  in  this  study  also  indicated  that  she  lacked 
subject  matter  expertise  and  expected  the  teachers  to  plan  the 
integration  of  the  program  into  the  existing  curriculum.    This  finding 
was  consistent  with  the  argument  raised  by  Becking  (1986)  and  Femian, 
Benedict,  and  Johnson  {1988)  that  practical  courses  on  working  with 
teachers  and  on  services  and  curriculum  development  are  essential  for 
the  school  media  specialist.    Without  teachers1  commitment  and  subject 
matter  expertise,  the  technical  expertise  of  the  media  specialist  alone 
did  not  warrant  the  program's  utilization,     one  possible  way  to  avoid 
reliance  on  the  single  change  agent  might  be  to  use  the  "multiplier 
mode"  proposed  by  Miles,  Saxl,  and  Lieberman  f  1388 )  .    The  strategy 
combines  the  expertise  of  subject  matter  teachers  and  the  expertise  of 
the  technical  specialist. 

This  study  illustrated  that  several  change  strategies  the  school 
media  specialist  used  might  have  affected  the  implementation  process. 
First,  the  media  specialist  tended  to  interact  more  with  people  who  had 
more  computer  knowledge.     Also,  her  use  of  technical  terminology 
appeared  to  distance    her  from  the  teachers.     Her  passive  approach  to 
<  providing  information  on  the  program  may  also  have  slowed  down  the  speed 
of  diffusion.     Strudler  (1991)  cautioned  that  an  effective  change  agent 
must  possess  strong  interpersonal  and  organizational  skills  to  carry  out 
necessary  training  and  support  functions.     These  skills  include 
initiative- taking,  securing  resources,  program  continuation,  and 
facilitating  group-functioning  and  decision.    Strudler  also  argued  that 
successful  change  tend  to  play  down  their  computer  expertise  and  resist 
being  called  "the  expert."    Rather,  they  prefer  to  be  seen  as  just 
another  teacher  who  knows  something  about  using  computers  in  school.  By 
de-emphasizing  their  expertise,  change  agents  could  minimize  the 
discrepancies  that  exist  between  them  and  teachers . 
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Implication:  An  innovation  *:hat  is  perceived  ec  be  complicated  ana  less 
educationally  significant:  may  Have  less  of  a  chance  of  being  implemented. 

The  study  showed  that  the  interactive  videodisc  program  involved 
the  combination  of  different  technologies  such  as  a  computer,  videodisc 
player,  CD-ROM,  and  television;  and  the  program  was  regarded  as  time- 
consuming,  complicated,  and  novel  by  the  teachers.    When  the  value  of 
"primary  sources-  was  not  a  consideration,  teachers  and  students  chose 
to  use  other  competing  resources  such  as  videotapes,  microfilms, 
newspapers,  and  magazines  which  were  perceived  as  easier  to  use.  Such 
findings  were  consistent  with  Rogers*   (1983)  caution  that  an  innovation 
will  have  a  higher  chance  of  being  adopted  if  it  is  perceived  to  have 
low  complexity,  high  relative  value,  high  trialability ,  high 
observability,  and  high  applicability. 

The  study  indicated  that  the  information  presented  through  the 
combination  of  technologies  was  contusing  to  users.     HyperCard  is 
considered  a  powerful  tool  with  non-linear  thinking  characteristics  so 
users  can  choose  their  own  learning  strategy  according  to  individual 
needs,    while  the  idea  of  giving  learners  control  of  their  own  learning 
may  have  a  strong  intuitive  appeal,  the  study  findings  did  not  show  that 
users  always  choose  or  appreciate  what  is  designed  for  them,     some  users 
were  impatient  about  the  wait-time  and  a  few  were  confused  by  the 
various  icons  presented  on  a  single-screen  display.    Some  teachers  were 
reluctant  to  organize  the  instruction  by  themselves.    Some  expected  the 
information  to  be  organized  in  a  package.    The  strong  interests  of 
students  in  watching  the  "execution  of  the  prisoner  who  assassinated 
President  McKinley-  in  the  program  was  regarded  as  entertaining  rather 
than  educational. 

These  reactions  are  consistent  with  literature  regarding  the 
carriers  to  using  the  interactive  video.    As  Jonassen  (1938}  and  Jost 
(1990)  pointed  out,  several  problems  occur  in  using  interactive  video 
technology:  difficulties  in  navigation  as  Che  user  gets  lost  in  the 
unstructured  knowledge  base;  difficulties  in  integrating  information 
into  personal  knowledge  structures  and  in  synthesizing  and  tuning  that 
knowledge;  and  cognitive  overload.    Attention  should  be  given  to  the 
design  and  development  of  interactive  videodisc  programs,  the  iconic 
metaphors  of  the  computer  screen  display,  and  the  instructional 
strategies  for  interactive  videodisc  programs. 

Researcher's  Reflection 

The  researcher  was  a  foreigner  in  the  United  States  and  an  out sidnr 
to  its  schools.     As  such,  the  researcher  was  impressed  that  teachers  and 
students  in  the  American  educational  system  were  able  to  have  access  to 
an  instructional  program  like  American  Memory.     However,  the  researcher 
was  also  impressed  that  the  implementation  has  so  little  direction  and 
that  the  program  was  not  really  utilized  by  the  teachers  and  students. 

Gathering  information  on  the  financial  and  human  resource 
investment  by  the  Library  of  Congress  in  developing  this  program  was 
outside  the  scope  of  this  study.    Nor  did  the  researcher  know  how  the 
decision  was  made  by  the  Library  of  Congress  to  develop  the  American 
Memory  program,    but  if  the  program  is  not  being  used  by  teachers  and 
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students,   it  cr*e  program  r.as  net  established  its  instructional 
significance,  then  questions  ntust  be  asked  about  why  tne  Library*  of 
Congress  developed  it  ar.d  who  has  really  benefited  from  this  interactive 
videodisc  program?    Perhaps  the  NHS  school  district,  NHS  itself,  and  the 
district  administrators  who  applied  for  the  grant  received  che  most 
benefit  ~-  promoting  their  public  and  personal  images.    Or,  perhaps  the 
Library  of  Congress  which  developed  the  program  shared  this  benefit.  If 
that  is  the  case,  if  school  administrators,  federal  officials,  and  their 
organizations,  buc  not  the  students,   realized  the  greatest  benefits, 
then  one  must  question  the  value  of  promoting  technology-related 
innovations  in  education. 
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Introduction 


A  perplexing  assortment  of  uses  of  the  term  " multimedia "  abounds  in  the  literature, 
variously  describing  computer-driven  devices,  computer  programs,  and  combinations  of  the  two, 
all  of  which  are  capable  of  producing  mixtures  of  sound,  graphics,  animation  and  video. 
Schroeder  (1991)  describes  "interactive"  multimedia  broadly  as  a  hybrid  technology  merging 
various  types  of  media  with  the  computer.  McCarthy  (1989,  p.  26)  provides  more  detail,  defining 
it  as  "the  integration  of  text,  audio,  graphics,  still  image  and  moving  pictures  into  a  single, 
computer  -  controlled,  multimedia  product."  Gayeski  (1993,  p.4)  defines  multimedia  in  terms  of 
systems,  as  "a  class  of  computer-driven,  interactive  communications  systems  which  create,  store, 
transmit,  and  retrieve  textual,  graphic,  and  auditory  networks  of  mfci-marion**,  while  Schwier 
and  Misanchuk  (1993,  p.  6),  to  place  the  emphasis  on  the  computer  program  rather  than 
hardware,  describe  interactive  multimedia  instruction  as  "an  instructional  program  which 
includes  a  variety  of  integrated  sources  in  the  instruction  with  a  computer  at  the  heart  of  the 
system/  Whatever  the  specific  definition,  such  systems  provide  the  possibility  of  nearly 
unlimited  combinations  of  digitized  audio,  graphics  and  text,  analog  and  digitized  video, 
accessed  through  the  computer  itself  and/or  a  variety  of  peripheral  devices  such  as  videodisc 
players,  compact  disc  players  and  music  synthesi2er.  They  have  the  potential  to  utilize  new 
developments  in  data-storage  (laserdisc,  compact  disc,  digitized  video  and  audio),  ever- 
increasing  computer  speeds  and  capabilities,  and  sophisticated  software  tools  to  allow  a  learner 
to  move  through  a  rich,  multimedia  base  in  a  way  that  fits  his/her  own  learning  needs  and 
preferences  (Schroeder,  1991). 

In teractive  multimedia  instruction  can  be  linear  and/or  structured  or  more  non-linear, 
resembling  a  database  incorporating  multiple  formats.  The  organization  of  the  information  in  a 
tightly  structured  program,  those  designed  for  a  specific  instructional  purpose  or  objective,  is 
generally  very  clear.  Programs  such  as  Eduquest's  Stories  and  More,  a  literature-  based 
curriculum  system,  offer  information  in  multiple  formats  (digitized  audio,  graphics,  text), 
allowing  the  learner  choices,  but  still  guide  the  user  in  a  structured  fashion.  However,  many  of 
the  newest  programs,  such  as  IBM's  Illuminated  Manuscripts  and  Intellimation's  Letter  from 
Birmingham  Jail  are  more  open-ended,  with  the  structure  of  the  information  less  obvious  to  the 
user.  These  resemble  multimedia  databases  where  the  user  selects  the  path,  information,  and 
format  to  view. 

Interactive  Multimedia  and  Instruction 

Romiszowski  (1990)  defines  instruction  as  a  goal-oriented  teaching  process  that  is  based 
on  pre-  planning  and  formative  evaluation.  He  views  hypermedia  [an  alternate  term  for  non- 
linear interactive  multimedia  instruction)  as  independent  of  specific  goals  and  instructional 
objectives,  as  possibly  a  component  of  an  instructional  system,  but  functioning  separately  as  an 
informational  system.  Duchastel  (1990)  claims  hypermedia  is  better  suited  to  informal  learning 
than  formal  instruction;  a  learning  rather  than  a  teaching  tor1.  Thus, "  [A]ny  structure  imposed 
upon  the  situation  in  order  to  enhance  the  learning  experier. to  be  derived  from  it  is  really 
extrinsic  to  hypermedia  itself4  (Duchastel,  1990,  p.  139).  Structure  in  hypermedia  may  actually 
inhibit  user  interest  and,  hence,  exploration.  Associational  learning  is  an  appropriate  use  of 
hypermedia. 

This  is  consistent  with  a  constructivist  view  of  learning  which  holds  that  the  learner 
individually  constructs  knowledge  through  interpreting  perceptual  experiences  of  the  external 
world  (Jonassen,  1991).  Learners  develop  unique  associations  between  prior  knowledge  stored  in 
long  term  memory  structures  and  new  information.  This  encoding  may  involve  the  integration 
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of  information  into  existing  schemata  (assimilation)  or  the  construction  of  new  schemata 
(accommodation).  Constructivists  encourage  inductive,  or  discovery,  learning  in  which  learners 
engage  a  domain  and  "construct  their  own  concepts  and  rules  based  on  their  interpretation  of 
the  instances  encountered"  (Rieber,  1992,  p.96).  Learning  occurs  through  interactions  with  one's 
environment  or  culture  and  the  potential  for  learning  increased  as  the  instructional  situation 
becomes  richer  and  more  engaging  for  the  learner.  From  a  constructive t  point  of  view,  it  is 
important  that  learning  environments  provide  learners  with  experience  in  and  appreciation  for 
multiple  perspectives,  encourage  multiple  modes  of  representation,  and  embed  learning  in 
realistic  and  relevant  contexts  which  support  intrinsically  motivating  and  self-regulated  learning 
(Rieber,  1992,  McMahon,  Carr  &  Fishman,  1993). 

Interactive  multimedia  instruction  can  provide  the  rich  learning  environment  central  to  a 
construed  vis  t  view  of  learning  by  furnishing  databases  of  information  in  multiple  formats  and 
perspectives  and  which  nurture  incidental  learning.  However,  such  environments  necessarily 
provide  a  limited  amount  of  structure  and  require  learners  to  create  their  own.  How  can 
designers  help  learners  to  construct  meaning  in  such  environments?  Learners  may  benefit  from 
strategies  that  help  them  encode  the  information  they  encounter  in  interactive  multimedia 
instruction,  strategies  such  as  paraphrasing,  generating  questions,  outlining,  cognitive  mapping, 
creating  images  and  summarizing.  These  activities  could  be  supplied  by  the  instructional  system 
or  generated  by  the  learner. 

Learning  Strategies  to  Facilitate  Encoding 

Learning  strategies  have  tended  to  be  explained  in  terms  of  information-processing 
theory  rather  than  constructivism.  Bruning  (1983,  p.  93)  defines  learning  strategies  as  "any 
internally  or  externally  mediated  cognitive  process  that  will  facilitate  transfer  of  information  to 
be  learned  from  short-term  into  long-term  memory".  Information-processing  theory  holds  that 
short  term  memory  has  a  limited  processing  capacity  such  that  learners  are  forced  to  select  from 
all  possible  information  presented  for  processing.  Short  term  memory  holds  information  for  only 
seconds  before  it  is  lost  or  encoded  for  storage  in  long  term  memory.  Learning  strategies  are 
generally  called  into  use  at  this  point  to  facilitate  the  transfer  of  information  (Bruning,  1983). 

Weinstein  and  Mayer  (1986)  also  link  learning  strategies  to  encoding.  Based  on  Cook 
and  Mayer's  (1983)  analysis  of  the  encoding  process,  they  describe  encoding  as  comprised  of 
four  components:  a)  selection  -  the  learner  actively  pays  attention  to  some  of  the  information 
impinging  on  sense  receptors  and  transfers  it  to  working  memory;  b)  acquisition  -  the  learner 
actively  transfers  information  between  working  and  long-term  memory  for  further  study;  c) 
construction  -  the  learner  actively  builds  internal  connections  between  ideas  in  the  information 
that  reaches  working  memory;  and  d)  integration  -  the  learner  actively  searches  long-term 
memory  for  prior  knowledge  and  transfers  it  to  working  memory  to  construct  external 
connections  with  the  new  information. 

Learning  strategies  are  used  to  rehearse,  organize,  and  elaborate  information  to  make  it 
more  meaningful.  Strategies  include  underlining  and  repetition  for  rehearsal;  outlining, 
categorization,  and  mapping  for  organization;  and  mental  imagining,  forming  analogies,  inserted 
questions,  paraphrasing  and  analyzing  key  points  for  elaboration.  Rehearsal  strategies  help  focus 
attention  on  important  information  and  encode  it  in  short  term  memory  (selection  and 
acquisition) ,  while  organization  strategies  help  in  selecting  appropriate  information  and 
constructing  connections  among  the  ideas  (construction).  Elaboration  strategies  help  transform 
information  by  making  the  material  more  meaningful  and  building  connections  among  new 
ideas  and  prior  knowledge. 
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This  paper  will  overview  the  research  on  perhaps  the  most  widely  used  learning 
strategy,  notetaking,  and  on  two  related  techniques,  outlining  and  concept  mapping.  It  will 
provide  an  analysis  of  their  effectiveness  to  aid  the  learner  in  encoding  new  information, 
examine  current  uses  of  these  strategies  in  interactive  multimedia  instruction,  and  offer 
suggestions  for  the  incorporation  of  these  learning  strategies  into  future  designs. 

Notetakingand  Related  Learning  Strategies 

Historically,  much  of  the  research  has  focused  on  notetaking  from  oral  or  videotaped 
lectures  (  Hartley  &  Davies,  1978,  Carrier  &  Titus,  1979,  Rickards,  1979,  Kiewra,  1985, 1987). 
Only  recently  have  studies  begun  to  focus  on  notetaking  from  text  (Kiewra,  DuBois, 
Christensen,  Kim  &  Lindberg,  1989;  Wade  &  Trathen,  1989).  Two  hypotheses  have  been 
advanced  to  explain  the  potential  effectiveness  of  notetaking  to  facilitate  learning:  1.  notetaking 
assists  encoding  or  2.  notetaking  provides  a  product  which  can  be  reviewed  later  (Divesta  & 
Gray,  1972,  cited  in  Rickards,  1979). 

The  encoding  hypothesis  or  process  function  of  notetaking 

This  hypothesis  holds  that  notetaking  is  beneficial,  independent  of  review,  because  it 
increases  attention  during  the  lecture  and  facilitates  encoding  of  lecture  ideas  into  long  term 
memory  (  Kiewra,  1985, 1987,  Hartley  &  Davies,  1978).  Peper  &  Mayer  (1986)  advance  three 
sub-hypotheses  to  explain  why  the  encoding  hypothesis  may  or  may  not  be  true.  The  first  two 
are  based  on  how  much  is  learned  while  the  third  focuses  on  the  degree  to  which  the  learner  is 
able  to  actively  relate  material  to  existing  knowledge.  The  first,  the  Attention  Hypothesis,  states 
that  notetaking  facilitates  learning  by  forcing  the  learner  to  pay  more  attention  to  presented 
material  or  to  process  presented  material  more  deeply.  Notetakers  would  perform  better  than 
non-  notetakers  on  dependent  measures.  The  second,  the  Distraction  Hypothesis,  holds  that 
notetaking  impedes  learning  by  forcing  the  learner  to  concentrate  on  the  motor  act  of  writing 
instead  of  more  fully  listening  to  lecture.  Notetakers  would  perform  worse  than  non-  notetakers 
on  dependent  measures.  The  third,  the  Generative  Hypothesis  (Wittrock,  1974),  is  based  on  idea 
that  notetaking  helps  the  learner  to  generate  meaning  by  relating  presented  information  to  prior 
knowledge  and  thus  building  a  more  integrated  learning  outcome.  Notetakers  are  expected  to 
perform  better  than  non-  notetakers  on  far  transfer  tasks  but  worse  on  near  transfer  tasks. 

The  external  storage  or  product  function  of  notetaking 

This  hypothesis  holds  that  notetaking  is  beneficial  because  the  notes  comprise  a 
tangible  product  which  can  be  retrieved  and  used  once  the  instructional  event  has  passed. 
Rickards  (1979)  suggests  two  possible  functions  for  the  external  storage  idea: 

1 .  )       a  Rehearsal  Function:  enhanced  recall  is  only  due  to  remembering  material  from 

the  notes  just  reviewed  (the  notes  provide  the  learner  with  more  information). 

2.  )       a  Reconstruction  Function:  recall  of  notes  allows  learners  to  reconstruct  parts  of 

the  passage  on  which  no  notes  previously  taken  (the  notes  help  the  learner  to 
recall  other  information). 
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The  effectiveness  of  notetaking 

The  encoding  hypothesis.  Research  evidence  for  this  function  is  mixed.  Combined 
findings  (Kiewra,  1987)  of  review  papers  by  Hartley  &c  Davies  (1978)  and  Kievvra  (1985)  reported 
35  studies  on  notetaking  from  lectures  supporting  the  encoding  function,  23  indicating  no 
significant  differences  between  those  who  do  and  those  who  do  not  record  notes  and  3 
indicating  the  activity  of  notetaking  to  be  dysfunctional  relative  to  listening  only.  For  notetaking 
from  text  material,  there  is  some  evidence  that  notetaking  served  a  minimal  encoding  function 
and  may  even  have  interfered  with  processing  (Kiewra,  DuBois,  Christensen,  Kim  &  Lindberg, 
1989).  Also,  a  study  by  Wade  and  Trathen  (1989)  indicated  that  noting  information  (including 
notetaking)  has  little  effect,  independent  of  the  importance  of  the  ideas  noted,  on  the  recall  of 
that  information. 

The  external  storage  hypothesis.  Initial  research  findings  appear  to  strongly  support  this 
function.  Combined  findings  (Kiewra,  1987)  of  review  papers  by  Hartley  &  Davies  (1978)  and 
Kiewra  (1985)  reported  24  studies  on  notetaking  from  lectures  supporting  the  product  function 
(those  who  reviewed  their  notes  achieved  more),  8  studies  indicating  no  significant  differences 
between  reviewers  and  non-  reviewers  and  no  study  indicating  reviewing  notes  to  be 
dysfunctional. 

The  rehearsal  function  of  the  external  storage  hypothesis.  More  recent  evidence  seems  to 
support  this  explanation  (Kiewra,  DuBois,  Christensen,  Kim  &  Lindberg,  1989;  Kiewra,  DuBois, 
Christian,  Mcshane,  Meyerhoffer  &  Roskelly,  1991).  These  investigators  indicate  that,  for  these 
earlier  studies,  what  has  traditionally  referred  to  as  "external  storage"  has  really  been  a 
combination  of  encoding  and  external  storage  because  the  latter  group  consisted  of  students 
reviewing  their  own  notes.  Learners  had,  in  effect,  two  chances  to  process  the  information  (a 
repetition  effect).  When  "external  storage"  is  reformulated  as  those  who  review  notes  but  who 
have  not  previously  viewed  the  lecture,  the  results  appear  less  conclusive.  In  these  two  studies, 
the  reformulated  product  function  was  shown  to  be  less  effective  for  factual  recall  and 
recognition  than  an  encoding  plus  storage  treatment  condition,  although  not  for  higher  order 
(synthesis)  performance.  Encoding  only  was  consistently  the  least  effective  treatment  and  no 
more  effective  than  listening  to  a  lecture  without  notetaking.  Thus,  notetakers  who  review 
outperform  notetakers  who  do  not  review. 

Further,  the  encoding  process  per  se  (without  review)  did  not  appear  to  be  aided  by 
recording  notes  on  linear  or  matrix  frameworks  (Kiewra,  DuBois,  Christian,  Mcshane, 
Meyerhoffer  &  Roskelly,  1991).  This  is  consistent  with  results  from  a  study  of  fill-  in  versus 
completed  graphic  organi2ers  (Kenny,  1992)  used  with  interactive  video  where  the  notetaking 
treatment  appeared  to  interfere  with,  rather  than  facilitate,  learning  from  the  program.  In  fact, 
when  learning  from  text  was  tested,  students  who  read  twice  from  the  material  but  did  not  take 
notes  outperformed  those  using  any  form  of  notetaking  (Kiewra,  DuBois,  Christensen,  Kim  & 
Lindberg,  1989).  Further,  Anderson's  (1980)  rcanalysis  of  a  study  by  Arnold  (1942)  found  that 
simply  rereading  a  passage  was  more  influential  on  recall  than  more  obtrusive  techniques  such 
as  outlining  which  he  hypothesized  might  actually  be  detrimental.  Other  studies  comparing 
notetaking  to  repetitive  reading  (Hoon,  1974;  Dynes,  1932;  Stordahl  and  Christensen,  1956)  found 
notetaking  no  different  from  reading  alone.  Notetaking,  whether  from  lecture  or  text,  may  be  a 
sufficiently  demanding  process  that  relatively  little  encoding  actually  occurs  during  the  act  of 
notetaking  (Kiewra,  DuBois,  Christian,  Mcshane,  Meyerhoffer  k  Roskelly,  1991;  Kenny,  1992). 
Also,  evidence  further  contradicting  the  efficacy  of  the  product  function  is  that  students' 
notetaking  is  generally  incomplete  (Kiewra,  1987,  Kiewra,  DuBois,  Christian,  Mcshane, 
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Meyerhoffer  &  Roskelly,  1991).  The  review  of  instructors's  notes  provides  best  results  (e.g. 
Kiewra,  1985,  Kiewra  &  Frank,  1988;  Risch  &  Kiewra,  1990)  although  student  notes  improve 
with  repetition  (Kiewra,  Mayer,  Christensen,  Kim  &  Risch,  1991). 

Outlining 

Outlining  is  defined  as  "a  high  level  skill  which  involves  identifying  relationships 
between  concepts  and  arranging  those  concepts  in  an  order  which  demonstrates  the 
superordinate  and  subordinate  nature  of  the  concepts  involved"  (Anderson-Madaus,  1990,  p.  3). 
Outlining:  a)  causes  focusing  on  important  points,  b)  helps  students  gain  familiarity  with  text 
structure,  c)  aids  retention,  d)  generates  useful  alternative  texts  to  supplement  materials  read, 
and  e)  causes  active  participation  in  learning  (Bianco  and  McCormick,  1989). 

The  effectiveness  of  outlining.  Several  studies  have  shown  that  outlining  results  in 
improved  recall  of  facts  (Barton,  1930,  Annis  and  Davis,  1975;  Glynn  and  DiVesta,  1977; 
Shimmerlik  and  Nolan,  1976).  The  use  of  hierarchical  summarization  strategy,  a  form  of 
outlining,  improved  comprehension  and  recall  in  the  middle  school  students  (Taylor  and  Beach, 
1984),  while  ninth-graders  who  completed  an  outline  grid  when  reading  did  better  on  multiple 
choice  recall  tests  (Slater,  Graves,  and  Piche,  1985).  Further,  research  asking  students  to  generate 
outlines  provides  some  support  for  their  facilitation  of  recall.  Outlining/  if  done  properly  (e.g. 
focusing  on  main  ideas,  organizing  from  abstract  to  concrete  in  the  hierarchy,  working  from  a 
classification  of  concepts)  and  not  viewed  as  being  too  intrusive  in  the  study  process,  can  aid  in 
both  organizing  new  information  and  integrating  new  knowledge  into  one's  personal  knowledge 
structure  (Hoffler,  1983;  Anderson,  1980).  For  example,  Tuckerman  (1993)  studied  coded 
elaborative  outlines,  a  method  which  involves  outlining  chapters,  coding  the  main  points  using 
a  six-code  scheme,  and  adding  elaborations  of  main  points,  such  as  examples  and  explanations. 
College  students  who  were  required  to  create  coded  elaborative  outlines  of  chapters  performed 
significantly  better  than  students  who  did  so  voluntarily,  who  created  standard  outlines,  or  who 
did  not  outline  (Tuckerman,  1993).  Students  given  the  option  of  outlining  generally  chose  not  to 
do  so,  but  those  not  required  to  write  these  outlines  performed  better  the  more  they  outlined. 

Outlining  has  also  been  compared  to  other  learning  strategies.  Palmarier  (1971)  found 
that  college  students  using  outlining  had  the  highest  level  of  essential  content  in  their  notes 
compared  to  those  using  a  three-column  method,  the  Bartush  Active  Methods  or  no  method, 
while  Snyder  (1984)  found  a  significantly  higher  recall  performance  for  the  outlining  method  in  a 
study  comparing  the  use  of  SQ3R,  outlining,  and  underlining  to  study  college  textbooks.  Iovino 
(1989)  found  that  outlining  significantly  helped  college  students  to  achieve  higher  immediate 
recall  than  did  networking,  but  networking  significantly  improved  their  ability  to  retain 
information  over  rime.  Students  were  given  five  hours  of  training  in  the  technique  they  used.  In 
a  study  of  the  use  of  outlining  and  clustering  in  prewriring  (Kellogg,  1990),  outlining  was  most 
beneficial  when  only  the  topic  was  provided  and  students  had  to  generate  and  organize  ideas, 
but  did  not  help  if  the  topic,  ideas,  and  organization  were  given. 

Othersrudies  report  no  advantage  of  outlining  over  other  strategies  (Arnold,  1942; 
Stordahl  and  Christensen,  1956;  Todd  and  Kessler,  1971;  Willmore,  1966  cited  in  lovino,  1989)  or 
a  marginal  advantage  for  outlining  in  writing  papers  (Branthwaite,  Trueman,  and  Hartley/ 1980; 
Emig,  1971).  Emig  (1975)  found  no  correlation  between  creation  of  an  outline  and  organization 
of  a  written  theme. 

Training  Needed  in  Outlining.  Anderson  (1980)  notes  that  outlining,  like  strategies  such 
as  imaging  and  paraphrasing,  requires  a  major  intrusion  in  the  reading  processes  and  also 
necessitates  a  significant  amount  of  training  to  use  properly.  In  research  on  student-generated 
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study  aids,  he  found,  that  without  explicit  instruction  in  the  use  of  the  aid,  students  do  as  well 
simply  rereading  the  assignment.  Outlining  is  often  one  of  the  only  strategies  taught  in  the 
middle  grades  and  is  the  strategy  preferred  by  students  (Bean,  Singer,  Sorter  &  Frazee,  1983). 
Training  to  outline  is  done  in  most  schools,  but  often  not  in  a  way  that  students  understand  it 
conceptually.  Matott  (1987)  claims  that  few  students  have  been  taught  that  outlining  requires 
the  identification  of  major  points,  then  all  sub-major  points  and  so  on.  Hoffler  (1983)  stresses  the 
need  to  present  an  outline  as  moving  from  abstract  to  concrete  as  one  goes  down  the  hierarchy. 
To  do  this  he  suggests  students  must  work  backwards,  classifying  from  concrete  items  to  more 
abstract  concepts. 

Spatial  notetaking  strategies 

An  outgrowth  of  schema  theory  (Kiewra,  1988)  is  the  study  of  learning  strategies 
involving  the  reorganization  of  linear  information  into  a  spatial  representation  that  specifies 
relationships  among  concepts.  The  process  of  creating  a  spatial  arrangement  requires  a  relatively 
deep  level  of  processing  aimed  at  determining  internal  connections  among  ideas.  Kiewra  &  his 
associates  (Kiewra,  DuBois,  Christensen,  Kim  &  Lindberg,  1989;  Risch  &  Kiewra,  1990;  Kiewra, 
DuBois,  Christian,  Mcshane,  Meyerhoffer  &  Roskeliy,  1991)  have  developed  and  tested  a 
modified  form  of  spatial  notetaking  strategy  which  supplied  learners  with  a  matrix  already 
containing  horizontal  and  vertical  headings  from  the  presentation.  Results  from  four  experiments 
varied.  The  investigators  found  no  significant  advantages  for  matrix  notetaking  in  three  studies. 
A  significant  difference  in  favor  of  the  technique  compared  to  conventional  notetaking  -  when 
students  were  alio  wed  to  review  their  notes  -  was  found  in  the  fourth  study. 

Concept  Mapping 

Perhaps  the  most  widely  known  and  researched  spatial  learning  strategy  is  concept 
mapping  (Novak,  Go  win  &  Johansen,  1983;  Novak  &  Go  win,  1984,  Heinze-Fry  &  Novak,  1990; 
Novak,  1990).  Concept  mapping  was  developed  as  a  spatial  knowledge  representation  technique 
based  on  Ausubel's  Theory  of  Meaningful  Learning,  a  theory  which  holds  that  knowledge  in 
memory  is  hierarchical,  with  more  general,  more  inclusive  concepts  subsuming  progressively 
less  inclusive,  more  specific  ones  (Novak  &  Gowin,  1984).  Concept  maps,  then,  are  drawn 
hierarchically,  with  more  inclusive  concepts  at  the  top  of  the  map  and  progressively  more 
specific  ones  arranged  below.  Concepts  are  placed  in  ovals  and  linked  by  labelled  lines  to  form 
semantic  units  termed  propositions. 

Concept  maps  are  viewed,  first  and  foremost,  as  a  tool  for  negotiating  meanings.  Maps 
anr  constructed  in  collaboratively  by  the  instructor  and  the  learners).  However,  it  can  also  be 
used  as  a  pre-instructional  tool  in  the  form  of  an  advance  or  graphic  organizer  and  as  a 
notetaking  technique  for  extracting  key  concepts  from  printed  or  oral  material  (Novak  k  Gowin, 
1984). 

The  effectiveness  of  concept  mapping.  A  recent  meta-analysis  of  nineteen  studies 
(Horton,  McConncy,  Ga llo,  Senn  &  Hamelin,  1993)  provides  an  overview  of  the  general 
effectiveness  of  the  technique.  Studies  had  to  occur  in  an  actual  classroom  and  use  concept 
mapping  as  an  instructional  tool  compared  to  an  alternate  technique  as  a  control.  Nearly  all 
studies  examined  involved  science  content,  material  which  could  be  argued  lends  itself  to  a 
hierarchical  depiction,  in  15  of  the  19  studies,  the  students  prepared  the  maps.  Effect  sizes  (E.S.) 
for  achievement  ranged  from  -0.31  to  +  2.02  with  a  mean  E.S.  of  0.46.  For  measures  of  student 
attitude  towards  the  particular  subject  matter,  E.S.'s  ranged  from  0.05  to  4.88  with  a  mean  E.S.  of 
1.57.  The  investigators  concluded  that  concept  mapping  has  generally  medium  positive  effects 
on  achievement  and  large  positive  effects  on  attitude. 
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Research  results  also  indicate  that  the  primary  benefit  of  concept  mapping  accrues  to  the 
person  who  constructs  the  map  (Novak,  1990;  Horton  et  al,  1993),  that  teacher-prepared  maps 
may  be  helpful  to  students,  but  only  after  they  have  had  practice  preparing  their  own  maps  and 
that,  at  first  (for  2-4  weeks),  there  is  generally  an  average  decline  in  performance  for  strategies 
that  require  meaningful  learning  although  they  finish  up  significantly  higher  (Novak,  1990).  The 
implication  is  that  time  is  needed  for  students  to  learn  and  learn  to  appreciate  meaningful 
learning  strategies  such  as  concept  mapping. 

Integrating  Learning  Strategies  in  Interactive  Multimedia  Instruction 

Interactive  multimedia  instruction  has  many  capabilities  to  facilitate  encoding.  It  can 
provide  a  variety  of  practice  strategies  with  feedback  depending  on  learner  choices,  provide 
learner  guidance  in  terms  of  recommended  paths,  incorporate  multimodal  techniques,  and 
provide  active  manipulation  of  lesson  content  for  interaction  and  resolution  of  conflicting 
information.  Current  interactive  multimedia  instruction  often  takes  the  form  of  a  multiple  format 
database.  In  many  systems,  learners  are  guided  to  manipulate  the  lesson  content  through 
notetaking.  The  learner  is  able  to  open  a  notetaking  screen  by  clicking  a  button,  type  in  notes 
about  the  current  topic  being  explored,  or  copy  sections  from  different  screens  into  the  notebook, 
and  either  print  or  save  these  notes  to  disk.  Guidance  on  taking  notes  is  rarely  given.  This 
presents  a  problem  since  learners  frequently  do  not  incorporate  structure  in  their  notetaking  or 
fail  to  elaborate  on  the  new  information.  Since  research  has  indicated  that  students  are  generally 
incomplete  note-takers  (Kiewra,  1987, 1988),  this  may  represent  a  serious  design  flaw.  A  number 
of  notetaking  techniques  have  been  proposed  which  could  be  incorporated  into  interactive 
multimedia  to  enhance  learner  encoding  in  a  more  structured  fashion.  These  include  linear 
approaches  like  outlining  (Kiewra,  DuBois,  Christian  &  McShane,  1988)  and  spatial  learning 
strategies  such  as  concept  maps  (Novak  and  Gowin,  1984). 

Programs  with  notebooks  or  notepads  are  often  fairly  rudimentary.  If  they  are  used  by 
students  they  risk  replicating  the  same  shortcomings  of  notetaking,  outlining  and  mapping  on 
paper.  The  computer  offers  capabilities  that  might  be  used  to  incorporate  the  best  aspects  of 
each  of  these  learning  strategies  while  avoiding  some  of  their  pitfalls.  It  also  offers  the  ability  to 
offer  a  degree  of  guidance  to  the  user  if  necessary  or  if  desired  by  the  instructor.  Capabilities 
that  might  be  exploited  in  incorporating  learning  strategies  into  a  notebook  include: 

-  cutting,  pasting  and  manipulating  text  and  media  (graphics,  sound,  video  -  e.g.  pasting 
in  Quicktime/  Video  for  Windows  movie  into  a  word  processing  document).  This  could 
be  done  in  outline  or  spatial  map  format  using  text  and  /or  icons. 

*         using  screen  titles  and /or  topics  to  organize  information  and  the  ability  to  connect  and 
manipulate  these,  pull  them  together  into  a  concept  map  or  hypermap.This  could  be 
done  in  outline  or  spatial  map  format  using  text  and/or  icons. 

►  creating  a  multimedia  document  from  information  gathered. 

Kozma  (1987)  has  also  proposed  the  electronic  notebook  idea.  Computers  can  aid  the 
learning  process  by  a)  supplementing  working  memory,  b)  making  relevant  prior  information 
available  quickly,  c)  prompting  the  learner  to  connect,  integrate,  and  structure  new  information 
with  old  in  an  easily  changeable  manner,  and  d)  providing  both  verbal  and  visual 
representations.  Learning  Tool  by  Intellimation  for  the  Macintosh  computer  is  an  example  of  a 
program  that  presents  the  learner  with  blank  workspaces  and  tools  to  develop  key  points, 
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connect  them  in  networks,  and  provide  textual  and  graphic  information  about  each.  The 
program  cues,  evokes,  models,  and  supplements  the  learner's  thought  processes,  in  effect,  it 
provides  a  tool  for  concept  mapping  and  related  spatial  notetaking  techniques. 

Outlining  in  a  computerized  environment  can  add  several  new  features.  Students  can 
incorporate  text  as  well  as  various  other  media  types  in  a  outline,  by  either  cutting  and  pasting 
or  linking  to  external  resources.  When  using  a  interactive  multimedia  program,  all  the 
information  can  be  included,  no  matter  what  the  format.  Applications  such  as  Mediatext, 
WordPerfect  and  Microsoft  Word  already  offer  these  features. 

Many  word  processors  also  offer  the  ability  to  expand  and  compress  outlines.  If  this 
feature  were  incorporated  into  a  program's  notebook,  students  could  view  their  information  at 
different  levels  of  the  hierarchy,  e.g.  degree  of  abstraction  or  detail.  If  the  designer  or  instructor 
desires,  he/she  could  provide  an  intact  outline  in  which  to  take  notes  and  include  media,  acting 
as  guide  through  the  system  and  showing  one  view  of  the  overarching  structure  in  a  linear, 
hierarchical  fashion.  Students  might  chose  to  view  or  not  to  view  this  as  desired.  Different 
outlines  might  be  available  to  show  different  perspectives  on  the  interactive  multimedia  system's 
information. 

Types  of  notebooks 

The  design  of  notebooks  for  interactive  multimedia  systems  can  incorporate  some  or  all 
of  the  features  discussed  previously.  Their  degree  of  sophistication  can  be  thought  of  as  a 
hierarchy  with  each  level  having  its  advantages  and  disadvantages.  A  possible  hierarchy  is 
described  below.  The  forms  of  learning  strategy  involved,  according  to  Weinstein's  and  Mayer  s 
(1986)  categories,  are  listed  in  parenthesis  after  each  point. 

1 .        Copy  and  paste  text  it  graphics  a  whole  screen  or  section  at  a  time  in  the  order  found. 
Advantages: 

a.  At  the  very  least  would  create  active  involvement  by  the  learner  in  the  selection 
of  relevant  information,  that  is  elicit  generative  learning  (rehearsal,  organ ization, 
elaboration). 

b.  Multiple  formats  could  be  included  in  the  notes  (rehearsal,  organization, 
elaboration). 

c.  The  accuracy  of  information  could  be  guaranteed  because  it  would  be  recorded 
exactly  (rehearsal). 

Disadvantages: 

a.  No  depth  of  processing  is  required  of  the  learner,  making  it  much  like 
underlining/highlighting. 

b.  The  learner  is  not  encouraged  to  rela  te  the  new  information  to  prior  knowledge 
and  to  elaborate  it  in  a  personally  meaningful  way. 

c.  Encourages  external  storage  of  information,  but  not  necessarily  encoding. 

d.  The  learner  cannot  show  relationships  among  ideas.  The  information  may  have 
to  be  stored  in  the  order  selected  and  copied  rather  than  reorganized  in  a 
personally  meaningful  way. 

e.  May  be  difficult  to  get  an  overview  of  the  organization  of  the  information. 
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Copy  and  paste  text  and  graphics  selected  in  the  order  found. 


Advantages: 

a.  Multiple  formats  could  be  included  in  the  notes  (rehearsal,  organization, 
elaboration). 

b.  The  accuracy  of  information  could  be  guaranteed  because  it  would  be  recorded 
exactly  (rehearsal). 

c.  The  learner  could  select  only  the  information  needed  as  it  is  found  (rehearsal, 
organization). 

d.  The  learner  is  more  actively  involved  in  selecting  information  to  include,  that  is, 
generative  learning  (rehearsal,  organization). 

Disadvantages: 

a.  The  learner  is  not  encouraged  to  relate  the  new  information  to  prior  knowledge 
and  to  elaborate  it  in  a  personally  meaningful  way. 

b.  Encourages  external  storage  of  information,  but  not  necessarily  encoding. 

c.  The  learner  cannot  show  relationships  among  ideas.  The  information  may  have 
to  be  stored  in  the  order  selected  and  copied  rather  than  reorganized  in  a 
personally  meaningful  way. 

d.  May  be  difficult  to  get  an  overview  of  the  organization  of  the  information. 


3.         Use  headings  in  interactive  multimedia  to  create  hierarchy  of  an  outline  (i.e.,  the  levels 
of  hierarchy  are  embedded  in  headings  themselves). 

Advantages: 

a.  This  would  accurately  show  the  hierarchy  intended  by  the  developer 
(organization). 

b.  If  the  learner  can  select  which  headings  to  include,  he  /  she  has  the  ability  to 
create  a  unique  structure  of  this  information  to  a  certain  extent,  that  is,  elicit 
generative  learning  (organization,  elaboration). 

c.  The  accuracy  of  the  information  could  be  guaranteed  because  it  would  be 
recorded  intact  from  the  program  (rehearsal). 

Disadvantages: 

a.  The  hierarchy  portrayed  may  not  match  the  learner's  prior  knowledge  or 
information  need. 

b.  The  learner  cannot  create  his  /  her  own  relationships  or  reorganize  the 
information  in  a  personally  meaningful  way. 

c.  The  learner  is  not  encouraged  to  relate  the  new  information  to  prior  knowledge. 

d.  Encourages  external  storage  of  information,  but  not  necessarily  encoding. 

e.  If  the  learner  cannot  select  the  headings  to  include,  superfluous  information 
could  be  included. 
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4.         Arrange  and  rearrange  text  and  graphics  copied  from  the  program  without  adding 
additional  information. 

Advantages: 

a.  The  learner  can  create  his  /  her  own  categorizations/connections  among  ideas 
(organization,  elaboration). 

b.  The  learner  can  display  his  /  her  own  conception  of  the  relationships  among  the 
ideas,  becoming  actively  involved  in  the  integration  of  new  information 
(organization,  elaboration). 

c.  Multiple  formats  could  be  included  in  the  notes  (rehearsal,  organization, 
elaboration). 

d.  The  accuracy  of  information  could  be  guaranteed  because  it  would  be  recorded 
intact  from  the  program  (rehearsal). 

Disadvantages: 

a.  There  is  no  inherent  way  to  show  relationships;  the  learner  must  use  text  to 
relate  the  ideas. 

b.  May  be  difficult  to  get  an  overview  of  the  organization  of  the  information. 


5.         Add  own  text/graphics  to  information  selected. 
Advantages: 

a.  The  learner  can  create  his/her  own  categorizations/connections  among  ideas 
(organization,  elaboration). 

b.  The  learner  can  display  his/her  own  conception  of  relationships  among  the 
ideas,  becoming  actively  involved  in  the  integration  of  new  information 
(organization,  elaboration). 

c.  Multiple  formats  could  be  included  in  the  notes  (rehearsal,  organization, 
elaboration). 

d.  The  learner  can  make  the  new  information  personally  relevant  through 
elaboration  with  prior  knowledge  (elaboration). 

e.  The  learner  can  connect  new  mformation  to  prior  knowledge  (elaboration). 

f.  The  learner  could  type  in  prior  knowledge  before  going  through  the  program, 
offering  a  framework  to  attach  new  information  to  (organization). 

g.  The  learner  or  instructor  could  provide  an  initial  framework  or  questions  to  act 
as  an  organizer  for  the  new  information  (organization). 

h.  The  learner  can  put  information  into  his/her  own  words,  facilitating  encoding 
and  retrieval  cues  (organization,  elaboration). 

i.  The  learner  can  add  connection  words  or  symbols  to  show  relationships  among 
the  concepts  (organization,  elaboration). 

Disadvantages: 

a.         There  is  no  inherent  way  to  show  relationships;  the  learner  must  use  text  to 
relate  the  ideas. 
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b.  May  be  difficult  to  get  an  overview  of  the  organization  of  the  information. 

c.  Information  added  may  be  inaccurately  copied  or  connected. 

6.        Create  an  outline  from  information  in  the  program  and  text/graphics  added. 
Advantages: 

a.  The  learner  can  create  his  /  her  own  hierarchical  arrangement  of  the  information 
(organization,  elaboration). 

b.  The  learner  is  actively  involved  in  selecting  information  to  be  included, 
elaborating  with  prior  knowledge,  and  integrating  new  and  prior  knowledge 
(organization,  elaboration). 

c.  Multiple  formats  could  be  included  in  the  notes  (rehearsal). 

d.  The  learner  could  type  in  prior  knowledge  before  going  through  the  program, 
offering  a  framework  to  attach  new  information  to  (organization). 

e.  The  learner  or  instructor  could  provide  an  initial  framework  or  questions  to  act 
as  an  organizer  for  the  new  information  (organization). 

f.  The  learner  can  put  information  into  his/her  own  words,  facilitating  encoding 
and  retrieval  cues  (organization,  elaboration). 

g.  It  may  be  easy  to  move  this  to  actual  student  production. 

Disadvantages: 

a.  Learners  not  trained  in  outlining  may  try  to  create  outline  in  a  linear  fashion  and 
not  really  develop  hierarchy. 

b.  The  outline  format  may  in  itself  be  cumbersome. 

c.  It  may  be  hard  to  for  the  learner  to  see  levels  of  hierarchy  unless  outline  can  be 
collapsed. 

d.  Not  all  information  lends  itself  to  a  hierarchy. 

e.  It  may  be  difficult  to  incorporate  other  media  resources  into  the  outline. 

f.  Information  added  may  be  inaccurate. 


7.        Creating  a  spatial  map  from  information  in  the  program. 
Advantages: 

a.  The  learner  can  create  and  see  the  hierarchy  or  connections  visually 
(organization,  elaboration). 

b.  The  information  included  does  not  have  to  be  hierarchical  in  arrangement 
(organization). 

c.  The  learner  may  be  able  to  hide  layers  of  hierarchy  as  necessary  (organization). 

d.  The  learner  may  be  able  to  create  linkages  among  concepts  and  describe  the 
nature  of  the  relationships  shown  by  the  links  (organization,  elaboration). 

e.  The  accuracy  of  the  information  could  be  guaranteed  if  it  was  recorded  intact 
from  the  program  (rehearsal). 
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Disadvantages: 


a.  Learners  not  accustomed  to  this  techniques  will  require  instruction  in  its  use. 

b.  Not  all  students  may  be  graphically  oriented. 

c.  Additional  information  cannot  be  added  to  make  it  personally  meaningful. 

8.  Spatial  mapping  with  learner  adding  own  text  elaborations. 

a.  The  learner  can  create  and  see  the  hierarchy  or  connections  visually 
(organization,  elaboration). 

b.  The  information  included  does  not  have  to  be  hierarchical  in  arrangement 
(organization). 

c.  The  learner  may  be  able  to  hide  layers  of  hierarchy  as  necessary  (organization). 

d.  The  leamer  may  be  able  to  create  linkages  among  concepts  and  describe  the 
nature  of  the  relationships  shown  by  the  links  (organization,  elaboration). 

e.  The  leamer  can  display  his  /  her  own  conception  of  the  relationships  among 
new  and  old  ideas,  becoming  actively  involved  in  the  integration  of  new 
information  with  prior  knowledge  (organization,  elaboration). 

f.  The  learner  could  type  in  prior  knowledge  before  going  through  the  program, 
offering  a  framework  to  attach  new  information  to  (organization). 

h.        The  learner  can  put  information  into  his/her  own  words,  facilitating  encoding 
and  retrieval  cues  (organization,  elaboration). 

Disadvantages: 

a.  Learners  not  accustomed  to  this  techniques  will  require  instruction  in  its  use. 

b.  Not  all  students  may  be  graphically  oriented. 

c.  Information  added  may  not  be  accurately  copied  or  connected. 

9.  Spatial  mapping  with  adding  own  text  /  graphics  elaborations  and  media  resources  from 
the  program. 

Advantages: 

a.  The  learner  can  create  and  see  the  hierarchy  or  conr ections  visually 
(organization,  elaboration). 

b.  The  information  included  does  not  have  to  be  hierarchical  in  arrangement 
(organization). 

c.  The  learner  may  be  able  to  hide  layers  of  hierarchy  as  necessary  (organization). 

d.  The  learner  may  be  able  to  create  linkages  among  concepts  and  describe  the 
nature  of  the  relationships  shown  by  the  links  (organization,  elaboration). 

e.  The  learner  can  elaborate  the  spatial  map  with  prior  knowledge  (organization, 
elaboration). 

f.  Multiple  formats  could  be  included  in  the  notes  (rehearsal,  organization, 
elaboration)). 

g.  The  learner  could  type  in  prior  knowledge  before  going  through  the  program, 
offering  a  framework  to  attach  new  information  to  (organization). 
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h.  The  learner  or  instructor  could  provide  an  initial  framework  or  questions  to  act 
as  an  organizer  for  the  new  information  (organization). 

i.  The  learner  can  put  information  into  his/her  own  words,  fecilita  ting  encoding 
and  retrieval  cues  (organization,  elaboration). 

j.         The  learner  can  add  connection  words  or  symbols  to  show  relationships  among 
concepts  (organization,  elaboration). 

Disadvantages: 

a.        Learners  not  accustomed  to  this  technique  will  require  instruction  in  its  use. 

b>        Not  all  students  may  be  graphically  oriented. 

c.         Information  added  may  not  be  accurately  copied  or  connected. 


Creating  a  multimedia  document  /  presentation. 
Advantages: 

a.  The  learner  can  create  his/her  own  categorizations/connections  among  ideas 
(organization,  elaboration). 

b.  The  learner  can  display  his/her  own  conception  of  relationships  among  the 
ideas,  becoming  actively  involved  in  the  integration  of  new  information 
(organization,  elaboration). 

c.  Multiple  formats  could  be  included  in  the  notes  (rehearsal,  organization, 
elaboration). 

d.  The  learner  can  make  the  new  information  personally  relevant  through 
elaboration  with  prior  knowledge  (elaboration). 

e.  The  learner  can  connect  new  information  to  prior  knowledge  (elaboration). 

f.  The  learner  could  type  in  prior  knowledge  before  going  through  the  program, 
offering  a  framework  to  attach  new  information  to  (organization). 

g.  The  learner  or  instructor  could  provide  an  initial  framework  or  questions  to  act 
as  an  organizer  for  the  new  information  (organization). 

h.  The  learner  can  put  information  into  his/her  own  words,  Eacilitating  encoding 
and  retrieval  cues  (organization,  elaboration). 

i.  The  learner  can  add  connection  words  or  symbols  to  show  relationships  among 
the  concepts  (organization,  elaboration). 

Disadvantages: 

a.  Learners  not  accustomed  to  this  technique  will  require  instruction  in  its  use. 

b.  There  is  no  inherent  way  to  show  relationships;  the  learner  must  use  text  to 
relate  the  ideas. 

c.  May  be  difficult  to  get  an  overview  of  the  organization  of  the  information. 

d .  Information  added  may  be  inaccurately  copied  or  connected . 
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Design  Considerations 


When  determining  the  type  of  notebook  to  include  with  a  program,  the  designer  much 
take  into  account  the  capabilities  of  the  computer,  the  needs  of  the  learners,  and  the  research  on 
notetaking  strategies.  Just  copying  and  pasting  in  notebook  is  not  likely  to  be  any  more  effective 
than  underlining  or  highlighting  in  a  text.  Research  has  shown  no  significant  effect  on  recall 
from  these  techniques.  Research  on  outlining  and  concept  mapping  offers  suggestions  for  the 
designer. 

Notetaking  in  an  interactive  multimedia  system  is  not  the  same  as  notetaking  in  a 
lecture,  because  pacing  not  an  issue  in  the  computer  program.  Learners  have  time  to  copy  text 
and  elaborate,  similar  to  underlining  with  margin  notes  or  two-column  notetaking,  and  then 
elaborate  with  their  own  text  and  graphics.  They  can  organize  or  categorize  information  as  they 
go  along  or  after  gathering  all  the  information,  due  to  the  ease  of  cutting  and  pasting.  They 
might  also  create  a  graphical  representation  of  the  information  as  they  go  along  or  after 
collecting  all  the  information.  Potential  ease  of  expanding  and  collapsing  outlines  may  make  it 
easier  for  learners  to  perceive  the  structure  of  the  information  at  various  levels.  Tuckerman's 
research  on  coded  elaborative  outlining  provides  a  basis  for  this  use  of  a  notebook  (Tuckerman, 
1993). 

Learners  also  need  to  have  reason  to  use  a  notebook.  Notetaking  and  other  learning 
strategy  research  has  shown  that  students  prefer  to  merely  read  and  reread  information  as  their 
strategy  of  choice.  Strategies  such  as  concept  mapping  and  outlining  may  not  be  in  their 
repertoire  of  strategies.  If  they  are  aware  of  the  strategy,  they  will  tend  to  use  those  they  feel 
most  comfortable  with,  even  if  it  may  not  be  the  best  for  the  situation.  A  notebook  may  be 
provided,  but  not  used.  A  notebook  that  provides  some  of  the  more  sophisticated  features  may 
need  introduction  to  the  students  or  they  may  use  it  in  a  simple  cut  and  paste  mode.  Such 
introduction  may  be  available  and  /or  guidance  in  the  use  of  the  notebook  provided  and  then 
extinguished  as  the  learner  becomes  more  comfortable  with  its  use. 

Notetaking  research  has  shown  that  leamer  who  elaborate  on  the  information  interact 
more  with  the  new  information  and  tie  it  to  existing  knowledge.  They  need  to  organize  and 
elaborate  on  information  to  help  in  encoding  and  creation  of  retrieval  cues,  tie  to  prior 
knowledge,  and  make  the  new  information  personally  relevant.  Guidance  to  elaborate  may  be 
required  in  a  notebook  where  this  feature  is  available.  The  combination  of  the  encoding  and 
external  storage  function  of  notebooks  is  most  valuable.  Learners  have  the  ability  to  review 
organized  and  elaborated  information  in  a  notebook.  This  allows  them  to  process  the 
information  again  and  assists  by  cuing  retrieval.  Generative  processing  research  supports  the  use 
of  a  notebook  to  copy  text,  graphics,  etc.  and  then  add  one's  own  elaborations.  Learners  can 
then  arrange  information  into  their  own  framework,  be  it  outline  or  spatial  map.  This  allows 
for  both  repetition  of  the  information  and  additional  processing. 

The  use  of  a  notebook  can  be  carried  even  further  to  creation  of  multimedia 
presentation.  This  allows  the  notebook  to  be  used  as  an  organizer  for  drafting  such  a 
presentation.  The  ideal  notebook  would  make  the  transition  to  a  finished  multimedia  product 
seamless. 

Another  use  of  the  notebook  wou.d  be  for  teacher  provided  questions,  outlines, 
frameworks,  or  keywords.  Any  of  these  strategies  can  act  as  an  advance  organizer  for  the 
learner,  especially  those  with  low  prior  knowledge.  They  can  provide  a  framework  for 
gathering  information  and  stimulate  recall  of  prior  knowledge.  Learners  could  also  be  asked  to 
generate  their  own  outline  or  framework  of  prior  knowledge  at  start  to  stimulate  recall  and 
provide  hooks  for  integrating  the  new  information.  Learners  could  also  generate  their  own 
questions  to  be  answered  at  the  start  and  fill  in  the  blanks  as  they  go  through  the  program. 
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No  matter  what  type  of  notebook  is  incorporated,  there  is  a  need  to  teach  use  of  the 
learning  strategy.  Students  using  even  simple  notetaking  have  difficulty  selecting  the  major 
concepts  and  including  the  appropriate  level  of  detail.  Many  simply  copy  verbatim  and  never 
elaborate  with  their  own  prior  knowledge,  a  strategy  proven  ineffective.  Outlining  and  concept 
mapping  are  even  more  challenging.  Unless  the  strategies  are  understood  and  carried  out 
correctly,  they  will  be  ineffective,  considered  burdensome,  and  probably  not  used. 
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Abstract:  Information  technology  has  developed  to  the  point  of  providing  a 
means  to  effectively  manage  nursing  and  related  health  care  data  for  nursing 
administrators,  educators,  practitioners,  and  researchers.  Therefore,  the  newly 
recognized  area  of  nursing  informatics  is  important  to  the  nursing  profession 
as  a  whole.  Nursing  schools  and  health  settings,  in  which  nurses  must  prepare 
for  or  function  in  their  nursing  roles,  must  fully  utilize  the  nursing  informatics 
capabilities  currently  available.  Technologically  enhancing  nursing  will 
ultimately  improve  client  care. 


Background 

The  amount  of  information  available  in  health  care  continues  to  grow  by  leaps  and 
bounds.  This  vast  quantity  of  information  makes  it  difficult  to  manage.  To  further 
complete  the  issue,  the  dissemination  of  this  information  is  not  timely  and  this  is  related  to 
the  time  gap  between  submission  and  Plication  of  journal  articles.  According  to  Werley 
and  Grier  (1981),  "the  information  with  which  nursing  is  concerned  is  broader  than  that  in 
other  health  care  fields.  If  nursing  is  to  continue  to  provide  patient  care,  can  it  afford  not  to 
emphasize  the  computerization  of  information?*  (p.  319).  Nursing  is  a  broad,  practice-based 
profession  and  needs  a  variety  of  information  to  be  readily  available.  Therefore,  nursing 
informatics  evolved— nursing's  integration  of  information  and  computer  science—to  enhance 
client  care  and  the  nursing  profession. 

However,  the  mountain  of  health  care  information  available  and  the  need  for  timely 
information  to  enhance  client  care  is  not  unique  to  nursing.  The  medical  field  has  recognized 
the  need  to  harness  the  information  explosion  and  facilitate  the  dissemination  of  relevant 
information.  The  National  Library  of  Medicine  (1986),  in  its  Report  of  Panel  4,  has  asserted 
that  "by  1996,  one  fifth  of  American  medical  schools  will  offer  training  in  medical 
informatics  and  all  American  health  science  schools  will  offer  training  in  medical  informatics 
as  part  of  their  curricula"  (p.  67).  As  a  means  to  this  desired  goal,  nursing  must  work 
collaboratively,  so  that  nursing  informatics  can  augment  the  health  data  field  in  such  a  way 
that  nurses'  skills  and  knowledge  enable  them  to  efficiently  and  effectively  manage  the  ever 
increasing  amount  of  health  care  data  available  today. 


Introduction 

Currently,  nurses  are  not  adequately  prepared  to  integrate  computer  and  information 
science  technology  (Romano,  1988).  The  result  of  this  technological  retardation  affects  every 
aspect  of  nursing.  Therefore,  nurses  must  be  able  to  understand  the  potential  uses  of 
computers  in  both  information  management  and  research  to  folly  benefit  them  in  their  work 
settings.  Nursing  must  be  able  to  move  forward  and  embrace  the  potentials  that  computers 
and  information  systems  offer. 


f\?Q  982 

I'  AJ  O 


Nursing  Informatics 


Nursing  informatics  means  so  much  more  than  simply  utilizing  computers.  It  is  the 
integration  of  computer  and  information  science  into  the  nursing  practice  settings  to 
ultimately  enhance  client  care  and  the  nursing  profession  as  a  jvhole.  Nurses  must  be  able  to 
manage,  manipulate,  and  dissrminate  data,  information,  and  knowledge.  Data  refer  to  raw 
facts  that  have  no  meaning  ii  And  of  themselves,  but  information  is  processed  data  that  is 
given  meaning.  It  is  important  to  note  that  each  user  has  a  knowledge  base  from  which  they 
draw  in  order  to  assimilate  information.  Knowledge  is  considered  to  be  a  body  of  related 
information.  However,  not  all  information  is  informative  to  everyone.  The  related  scraps  of 
information  are  integrated  and  synthesized  to  form  a  knowledge  base  for  the  individual. 
Therefore,  nursing  informatics,  as  adapted  from  McGonigle  and  Eggersf  (1991)  definition,  is 
the  synthesis  of  nursing  science,  information  management  science,  and  computer  science  to 
enhance  the  input,  retrieval,  manipulation,  and/or  distribution  of  nursing  data,  information, 
and  knowledge  for  the  four  areas  of  nursing  to  enhance  client  care  while  advancing  the 
profession  of  nursing. 

Nurses  must  integrate  nursing  informatics  into  their  professional  lives.  When  nurses  in  all 
four  areas  of  the  nursing  profession  have  a  firm  nursing  informatics  knowledge  bas^  they 
will  so  longer  have  to  rely  solely  on  commercially  available  nursing  systems  but  may  be  able 
to  escalate  to  the  level  of  designing  and  developing  their  own  nursing  systems  to  meet  specific 
needs. 


Nursing  Administration 

The  nursing  administrator  is  responsible  for  coordinating,  supervising,  and  facilitating 
subordinates  to  work  toward  the  goals  and  objectives  of  the  institution  in  which  they  practice. 
They  also  integrate  research  findings,  conduct  research  studies,  and  assist  with  research 
efforts  under  their  governance.  They  are  also  educators  at  times,  as  well. 

Administrators  are  able  to  access  more  than  150  functions,  depending  on  the  system  used 
in  their  practice  facility  and  these  functions  range  from  budget  preparation  to  personnel 
surveys  and  beyond.  Additionally,  they  have  access  to  quality  assurance  worksheets  and 
collation. 

Nurses  :s  these  positions  have  a  variety  of  uses  for  their  electronic  communications  that 
range  from  administrator  to  administrator,  administrator  to  nursing  employee,  administrator 
to  patient,  administrator  to  physician,  and  administrator  to  other  health  care  employees. 
Furthermore,  they  are  able  to  network  within  their  facility,  across  the  state,  country  or  the 
world,  and  they  can  access  electronic  bulletin  boards  and/or  even  create  their  own. 
Administrators  can  electronically  manage  their  personnel  files,  budgets,  payroll,  census,  staff 
planning,  staff  allocation,  and  summary  reports.  With  the  move  toward  the  concept  of  case 
management,  they  can  develop  the  critical  pathways  and  electronically  manage  this  data  and 
information.  As  if  this  integration  is  not  enough,  word  processing,  spread  sheets,  and 
database  management  programs  can  further  facilitate  their  rues. 

Nursing  administrators  must  support  the  integration  of  nursing  informatics  into  their 
settings  to  enhance  the  nursing  profession.  They  could  also  benefit  from  the  integration  of 
nursing  informatics  into  their  specialty  area  of  nursing.  Nursing  administrators  are  able  to 
electronically  manage  their  data,  including  personnel  and  client  data,  and  the  electronic 
networking  capabilities  that  link  them  with  other  administrators  and  promote  collaboration. 
Since  administrators  are  responsible  for  all  four  areas  of  nursing,  they  must  assume 
responsibility  for  the  integration  of  nursing  informatics  into  each  area.  It  will  be  their 
commitment  to  the  increased  integration  and  exploration  of  technological  advances  that  will 
enhance  nurses'  job  performance  and  the  advancement  of  the  nursing  profession. 
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Nursing  Education 

Nurse  educators  impart  knowledge  and  skills.  These  nurses  also  incorporate  research 
findings  and  conduct  and  assist  with  research  studies.  There  has  also  been  an  increase  in  the 
number  of  nursing  educators  that  are  currently  practicing  as  well. 

Within  recent  years,  the  health  care  field  has  experienced  a  growth  in  the  number  of 
facilities  utilizing  personal  computers  to  meet  their  inservice,  staff  education,  and  patient 
teaching  needs.  These  facilities  have  also  integrated  client  discharge  educational  plans.  As  a 
result,  the  nurse  educator  can  use  computer  assisted  video  instruction  (CAVI),  computer 
assisted  instruction  (CAI),  compact  disc  interactive  (CDI),  computer  mediated  instruction 
(CMI),  expert  systems,  and  knowledge  synthesizers.  Thus,  the  nurse  educator  preparing 
nursing  students  must  be  informatics  competent  to  fully  prepare  their  students  for  the  real 
world  of  the  registered  nurse  through  the  use  of  miiitjnwtia  and  computerized  testing. 
Additionally,  the  nurse  educator  needs  to  both  demonstrate  and  utilize  a  variety  cf 
information  systems  that  the  students  might  encounter  during  their  clinical  rotations  and  on 
the  Job  after  graduation  More  importantly,  nurse  educators  must  require  nursing  students  to 
be  informatics  competent  and  capable  of  maneuvering  within  an  electronic  environment  in 
order  to  conduct  electronic  literature  searches  and  navigate  within  electronic  bulletin  boards 
and  electronic  mail  systems. 

Nursing  education  must  assess  the  nursing  informatics1  needs  of  the  nursing  profession 
and  make  necessary  auricular  changes  to  better  prepare  their  students  because  the  entry  level 
professional  can  no  longer  be  technologically  retarded.  This  beginning  practitioner  needs  to 
be  informatics  competent  Nursing  schools  are  not  adequately  preparing  their  students  for 
computer  use,  and  nurses,  who  are  currently  working  and  want  to  learn  about  nursing 
informatics,  are  not  afforded  continuing  education  course  work  in  this  area.  Therefore, 
nursing  educators  must  begin  to  incorporate  and  utilize  nursing  informatics  throughout  their 
nursing  curriculums. 

Nursing  Practice 

Nursing  practice  refers  to  nurses  who  provide  direct  care  to  the  client,  coordinate  and 
supervise  subordinates  who  are  also  providing  direct  client  care,  and  oversee  the  education  of 
peers,  subordinates,  and  clients.  This  nursing  area  focuses  on  health  promotion, 
maintenance,  early  detection,  and  intervention  Furthermore,  practicing  nurses  integrate 
current  research  findings  and  assist  with  clinical  research  efforts. 

The  nurses  who  are  practicing  at  the  client's  side  are  using  computers  and  information 
science  to  manage  their  client's  electronic  data.  They  also  need  timely  accurate  information 
to  enhance  their  nursing  skills  and  client  care.  The  use  of  electronic  bulletin  boards, 
databases,  and  networking  with  other  professionals  will  greatly  enhance  their  ability  to  access 
needed  information.  Utilizing  nursing  informatics,  practicing  nurses  would  be  able  to 
retrieve  up  to  date  information  electronically  rather  than  waiting  for  the  journal  article  to  be 
published  because  by  publication  date,  the  information  may  be  outdated.  Thus,  practicing 
nurses  can  greatly  improve  their  client  care  by  having  access  to  electronic  information  and 
networking.  The  networking  capabilities  bam  linked  nurses  across  the  country  and  beyond. 
This  high-tech  communication  capability  facilitates  collaboration  and  clinical  research,  an 
area  of  priority  since  nursing  is  a  practice-based  profession. 

In  their  direct  care  role,  the  use  of  automated  nursing  notes  has  already  increased.  These 
notes  contain  nursing  orders  that  require  medical  orders,  as  well  as  independent  nursing 
functions  including  the  nursing  process  documented  on  the  cue  plan.  As  the  nursing 
profession  moves  toward  case  management,  critical  pathways  will  be  electronically 
documented,  monitored*  and  researched.  There  are  many  functions  that  have  become 
electronic  including  the  nursing  process,  transfer  notes,  and  discharge  plans  and  summaries. 
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Nursing  Research 


The  nurse  researcher  is  looking  for  solutions.  This  nurse  can  also  be  an  educator  or 
currently  practicing.  A  nurse  researcher  uses  information  retrieval  methods  that  are  vastly 
improved.  Electronic  media  are  timely  and  provide  the  researcher  with  immediate  data, 
information  and  knowledge.  Fnhannrd  data  processing  methods,  datitase  management 
systems,  statistical  analyses,  graphic  displays,  and  text  editing  capabilities  facilitate  the 
research  effort  even  more.  An  example  of  this  can  be  seen  with  word  processing  packages 
that  place  text  into  manuscript  style  with  specific  style  formats  available. 

As  on-line  information  sources  continue  to  develop  and  become  refined,  data,  information, 
and  knowledge  can  be  available  within  minutes  of  its  submission.  Nursing  researchers  will 
be  able  to  quickly  disseminate  their  findings  using  electronic  formats  and  will  also  be  able  to 
electronically  locate  current  and  accurate  information  as  more  bulletin  boards,  newsletters* 
and  journals  move  toward  becoming  available  on  INTERNET. 

Electronic  networking  capabilities  assist  nursing  researchers  by  Unking  them  in 
co-  ununication  with  other  researchers  who  share  commonalties.  This  enhances  collaboration 
and  joint  research  ventures.  Coupling  the  networking  capabilities  with  the  ability  to  access, 
retrieve,  input,  and  manipulate  data  greatly  promotes  nursing  research  efforts.  The 
immediate  dissemination  of  research  findings  will  enhance  the  nursing  profession  as  a  whole. 


Summary 

Each  of  the  four  areas  of  nursing  can  benefit  from  using  nursing  informatics  and  the 
functions  ore  shared  across  the  four  areas.  That  is,  nurses  wto  are  providing  direct  care  use 
functions  once  thought  to  be  only  for  research,  administration,  and/or  education-  In  some 
instances,  nurses  are  now  evolving  from  users  to  developers  of  systems  designed  specifically 
to  meet  nursing  needs.  They  are  no  longer  looking  to  others  to  design  systems  to  solve  their 
problems.  Instead,  they  are  developing  the  systems. 

Nursing  informatics  must  come  of  age;  it  is  important  to  meet  the  challenges  currently 
facing  the  nursing  profession  while  preparing  for  nursing's  future.  As  the  nursing  profession 
embarks  on  incorporating  the  state-of-the-art  electronic  capabilities  into  nursing  informatics, 
it  must  also  project  and  embrace  the  future  direction  of  the  technological  explosion.  Nurses 
must  be  informatics  competent  to  continue  to  grow  professionally  while  fostering  the  growth 
of  the  nursing  profession. 
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This  study  investigated  four  types  of  media  formats:  talking  head,  voice-under- 
text,  voice-under-video  and  muttiformat  (a  combination  of  talking  head,  voice-under- 
video  and  text).  Sixty-four  suburban  middle  school  students  were  shown  four  different 
narrative  stories:  each  story  was  presented  to  the  students  in  a  different  media  format 
The  four  stories  were  exactly  the  same  except  for  the  format  of  presentation.  The  students 
were  tested  to  determine  what  thev  remembered  about  each  story  through  verbal  recall 
and  a  spatial  task.  The  verbal  recall  results  in  this  study  indicated  that  memory  of  a 
narrative  differed  significantly,  depending  on  whether  it  was  shown  in  a  talking  head, 
voice-under-text  voice-under-video  or  muttiformat  presentation.  More  particularly,  the 
formats  of  presentation  which  appeared  to  most  enhance  memory  of  a  narrative  were  the 
primarily  visual  formats  (muttiformat  and  voice-under-video).  Students  were  less  apt  to 
remember  stories  presented  in  the  primarily  audio  formats  (voice-under-text  and  talking 
head).  In  contrast  initial  analysis  of students'  spatial  recall  was  inconclusive.  However, 
we  argue  that  this  was  due  more  to  the  inadeauacyofthe  soatial  scoring  schemes  to 
reflect  the  real  differences  in  the  drawings.  This  paper  reports  the  verbal  recall  findings  of 
the  study  and  suggests  how  the  spatial  task  findings  can  be  reviewed  and  evaluated  to 
further  an  understanding  of  the  relationship  of  format  of  presentation  to  understanding. 


INTRODUCTION 

A  multimedia  presentation  combines  the  inherent  media  attributes  of  different 
electronic  technologies  to  produce  a  message  which  uses  the  auditory/verbal  and 
visual/oictorial  channels  of  information  processing.  Just  as  the  choice  of  a  medium  is 
important  to  how  a  message  is  received  and  understood,  so  too  is  the  choice  of  the  format 
of  presentation  within  that  medium  because  of  the  possible  effects  it  may  have  on  the 
perception  and  ultimate  understanding  of  the  message.  Research  on  the  interplay 
between  the  auditory/verbal  and  visual/pictorial  channels  of  information  processing 
suggests  that  understanding  of  electronic  media  presentations  is  determined  not  only  by 
the  clarity  of  the  message  being  presented,  but  also  by  the  associations  made  between  the 
two  cognitive  channels  during  processing. 

There  have  been  many  investigations  comparing  children's  understanding  of  visual 
and  verbal  information  presented  in  television  programs.  Some  have  concentrated  on 
defining  the  distinctions  between  iconic  and  symbolic  modes  of  representation  (Bruner 
1 966;  Glass,  Holvoak  ft  Santa.  1 979).  Other  research  has  concentrated  on  the 
behavioral  issues  associated  with  visual -auditory  presentations  ,  such  as  vocal 
characteristics  (Shosteck,  1 973;  Burgoon,  1 978;  Beighley,  1 952;  Hadwiger,  1 970: 
Smith  &  McEwen.  1 973)  or  eve  contact  with  the  camera  lens  (T  ankard,  1 970). 

The  inherent  characteristics  of  a  newscast  itself  also  have  been  found  to  influence 
a  viewer's  perception.  For  example,  viewers  are  more  apt  to  believe  (Baggaiey,  1 980). 
agree  with  (Allen,  1 973),  or  recall  unfamiliar  information  (Davey  ft  Kapinus,  1 985) 
presented  first  in  a  newscast.  The  pace  or  action  of  a  televised  presentation  also  appears 
to  influence  understanding:  children  are  less  aot  to  understand  and  recall  televised 
information  that  is  presented  through  dialogue  than  content  presented  with  moderate  to 
high  rates  of  character  action  (Calvert  et  al.,  1 982;  Hayes  ft  Birnbaum,  1 980;  Watkins, 
Calvert,  Huston-Stein  ft  Wright,  1 980).  On  the  other  hand,  increasing  the  video  pacing  of 
a  film  does  not  appear  to  affect  recall  (Schlater,  1 970).  It  has  also  been  shown  that  there 
is  a  relationship  between  the  visual  and  verbal  memory  of  televised  messages  and  the 
emotional  valence  of  that  message  (FIske,  1 982;  Bradley,  Greenwald,  Petry  ft  Lang, 
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1 992:  Bower,  1 981 ).  The  left  hemisphere  of  the  brain  is  associated  with  positive 
messages  and  the  right  side  of  the  brain  is  thought  to  process  negative  messages  best 
(Lang  &  Friestad.  1 993;  Davidson.  1 985).  Furthermore,  since  it  is  generally  believed  that 
the  left  hemisphere  of  the  brain  is  the  verbal  processor  and  the  right  hemisphere  is  the 
visual-soatial  processor  (Kinsbourne,  1 982).  there  have  been  suggestions  that  there  is  a 
relationship  between  the  valence  of  the  message  and  it's  format  of  presentation  with 
regard  to  the  memory  (Reeves.  Lang.  Thorson  and  Rothschild.  1 988). 

A  tremendous  amount  of  television  research  has  focused  on  what  Salomon  (1 974) 
refers  to  as  the  'inherent  symbolic  media  attributes'.  In  general  these  media  attributes 
can  be  defined  independently  of  the  narrative  content  (Coynil.  1 974).  For  example,  cuts 
have  been  shown  to  increase  interest  in  a  film  (Kraft.  1 986)  and  serve  as  visual 
punctuation  (Carroll  &  Bever.  1 9  76)  as  well  as  a  method  of  setting  the  pace  of  the  film 
(GiannettL  1982). 

Viewer  comprehension  of  an  electronic  media  presentation  is  influenced  by  manv 
factors  and  interactions  including  how  the  information  is  organized  and  linked  to 
knowledge  structures  in  long  term  memory.  In  general,  these  frames  (Minsky.  1 975); 
scriots  (Shank  and  Abelson.  1 977);  or  schemata  (Anderson.  Spiro  and  Anderson,  1  978). 
as  thev  are  variously  referred  to  in  the  literature,  are  used  to  facilitate  a  subject's  ability  to 
make  inferences  about  a  concept.  Some  researchers  believe  that  to  understand  the 
influence  electronic  media  has  on  learning  it  is  important  to  understand  the  skills  needed  to 
process  the  form  of  a  message  as  well  as  its  content  (Clark.  1 975:  McLuhan,  1 964.  Oison 
&  Bruner,  1 9  74).  These  skills  include  visual  and  verbal  analysis,  perspective  taking  and 
soatial  manipulations.  The  media  attributes  which  affect  knowledge  acciuisftion  are  not 
limited  to  the  physical  representation  of  the  message.  When  an  individual  is  presented  with 
information  about  an  event,  a  different  set  of  rules  will  be  applied  if  that  event  is  read  in  a 
newsoacer  rather  than  presented  on  radio  (Williams.  Paul  4  Ogilvie.  1 957:  Wilson.  1 974). 
or  if  the  message  is  seen  on  television  in  a  taikina  head  format,  rather  than  in  a  video 
format  (Katz.  Adoni.  et.aL.  1 977;  Edwardson.  Kent  4  McConnell.  1985).  Furthermore, 
research  suggests  that  there  are  hemispheric  differences  in  information  recalled  from 
global  versus  local  processing  (Hansen.  1 98 1 )  or  positive  versus  negative  emotions 
(Davidson,  1985). 

The  effect  of  media  attributes  on  the  message  is  perhaps  most  pronounced  in 
multimedia  productions  because  several  electronic  media  forms  are  combined  to  create 
one  presentation.  The  attributes  of  one  medium  interact  with  those  of  another  and  it  is 
these  interactions  that  ultimately  form  the  attributes  of  multimedia.  Therefore,  it  is 
necessary  to  analyze  the  parts  of  a  multimedia  presentation,  such  as  format  of 
presentation,  to  define  the  whole. 

The  formats  of  presentation  examined  in  this  experiment  included  those  most  often 
used  in  multimedia  productions;  talking  head,  yoice-under-text,  voice-under-videa  and  a 
multiform  at  combination  of  all  three.  These  formats  are  familiar  to  students  as  thev  are 
frequently  used  individually  or  in  tandem  in  all  electronic  media  presentations.  The 
Question  asked  in  this  experiment  was  a  simple  one:  would  children  remember  more  about 
a  narrative  if  it  was  presented  in  one  format  rather  than  another.  Specifically,  when 
subjects  were  given  simultaneous,  or  near  simultaneous  signals  from  multiple  sources. 
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how  were  those  stimuli  encoded  bv  the  subject  to  be  either  stored  and  utilized  in  long  term 
memory,  or  discarded  as  irrelevant? 


Method 
Subjects 

Sixiv-four  students  from  Westchester  County,  New  York  participated  in  the 
experiment.  The  32  bovs  and  32  girls  participating  in  this  experiment  ranged  in  age  from 
1 2  vears  7  months  to  1 5  years  1  month  and  were  racially  and  ethnically  mixed  to  include 
5 1  White.  5  African-American,  4  Latino  and  4  Asian  children.  English  was  the  first 
language  for  all  the  students.  The  students  were  evenly  distributed  to  view  the  four 
sequences  of  the  experiment:  eight  boys  and  eiaht  airis  saw  each  story  once  and  each 
media  format  condition  once. 


'  Materials 
Formats  of  Presentation 

Each  story  was  produced  in  four  formats  of  presentation:  talking  head,  voice- 
under-text.  voice-under-video  and  muttiformat.  which  is  a  combination  of  the  three.  The 
talking  head  was  taped  in  a  studio  with  a  noninterf  ering  blue  backdrop.  A  female 
announcer  read  from  cue  cards  placed  near  the  camera  lens  to  replicate  the  style  of  most 
video  announcers.  She  did  not  appear  to  be  reading  the  story  from  a  text  and  she  looked 
straight  into  the  lens  to  increase  believabilttv  and  influence  recall  (Tankard.  1 970).  The 
pace  of  the  oresentation  was  between  1 60  and  1 80  words  per  minute,  within  the  ideal 
range  for  memory  (Schiater  (1 970).  The  soundtrack  from  this  initial  taping  was  used  for 
the  other  three  presentation  formats.  Thusf  the  audio,  including  vocal  inflections  and 
timing,  was  consistent  across  all  four  experimental  conditions. 

The  voice-under-text  v  ^sd  in  the  experiment  was  designed  to  suggest  an 
interactive  computer  screen.  1  he  text  was  shown  in  serif  font  with  white  letters  on  a  blue 
background.  Each  "page*  was  numbered  at  the  bottom  of  the  screen  and  there  were  ten 
lines  of  text  on  each  'page*.  This  was  in  keeping  with  the  recent  literature  on  screen 
design  (Faiola  and  DeBloois,  1 988). 

The  voice-under-video  format  was  produced  by  first  laying  down  the  soundtrack 
on  a  3/4*  tape.  Video  which  was  redundant  with  the  audio  of  the  story  was  then  copied 
onto  the  soundtrack.  For  example,  if  the  text  described  a  man  at  work  on  a  sewing 
machine,  a  video  showing  a  man  working  at  a  sewing  machine  was  shown. 

The  final  format  used  In  the  experiment  was  produced  by  combining  talking  head 
with  the  voice-under-video  and  then  enhancing  the  film  with  key  poir* •  of  the  story  by 
using  captions  over  the  video.  These  captions  were  chosen  from  the  main  headings  of  a 
conceptual  map  developed  from  the  text. 
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Each  of  the  testing  materials  designed  and  used  for  this  studv  were  unique:  each 
of  the  formats  of  presentation  had  their  own  individual  characteristics,  and  each  of  the 
narratives  were  organized  in  a  different  structure  or  with  a  different  purpose.  The  formats 
of  presentation  were  categorized  as  being  either  primarily  audio  or  primarily  visual.  The 
talking  head  and  voice-under- text  were  primarily  audio  because  the  main  focus  of  the 
viewer  of  a  presentation  in  either  of  these  formats  was  on  the  audio  portion  of  the 
narration.  In  other  wordr .  the  visual  representation  of  the  story  through  text  on  the  screen 
provided  additional  information,  but  was  not  the  primary  focus  of  the  viewer's  attention. 
Similarly,  the  eve  contact  of  the  narrator  may  have  provided  an  additional  incentive  for  the 
viewer  to  listen  to  the  story,  but  no  additional  information  about  the  narration  was 
projected. 

The  primarily  visual  formats  identified  in  this  studv  were  the  voice-under-video  and 
muitiformat  presentations.  In  each  of  these  formats,  the  primary  focus  of  the  viewer  was 
the  image  on  the  screen.  Admittedly,  the  audio  portion  of  the  film  provided  a  structure  to 
the  images  being  projected  on  the  screen,  but  it  was  the  visual  image  which  seemed  to 
enhance  cue  recognition  for  information  processing  and  retrieval,  in  the  voice-under- 
video  presentation  there  was  no  break  in  the  visual  images  being  projected  over  the  audio 
narration.  This  was  not  the  case  in  the  muitiformat  presentation  because  the  video  portion 
was  interspersed  with  clips  of  the  talking  head  and  captions  of  text.  Theoretically,  the  two 
formats  should  compliment  each  other  by  directing  and  maintaining  the  attention  of  the 
viewer. 


The  Stories 

In  aeneral.  understanding  and  rememberinq  the  events  of  a  dramatic  story  utilize 
the  same  processes  people  empower  to  understand  and  recall  other  events  and 
presentations  (Bower  and  Black.  1 980).  Since  narrative,  dramatic  stories  have  proven  to 
be  valuable  In  testinq  children's  understanding  of  media  presentations  (Meringoff , 
et.aL  1 983).  four  original  narratives  were  written  for  this  experiment.  The  stories  were 
Intentionally  diverse  to  eliminate  any  confusion  when  subjects  were  asked  to  recall  the 
presentation.  The  four  narratives  were:  'A  Child  is  Taken'.  'The  Strength  of  Memories 9 . 
'Return  to  the  Sea'  and  The  Sound  of  Hatred'. 

*A  Child  is  Taken',  was  about  a  young  child  being  kidnapped  from  her  homo  In  thu 
middle  of  the  night.  Time  and  space  were  kev  factors  in  understanding  the  Inference 
presented  In  this  narrative  which  sequentially  presented  the  events  surrounding  the 
kidnapping  and  vividly  described  the  scene  of  the  crime.  A  second  story,  entitled  'The 
fllrength  of  Memories',  was  about  a  prisoner  in  Auschwitz  during  WW!I.  It  was  presented 
as  a  series  of  abstract  recollections  bv  a  man  confined  to  a  German  priion  cell*  There  was 
nothing  sequential  or  predictable  about  the  story  as  it  described  the  man's  girlfriend,  his 
family,  and  his  past  career  as  a  champion  boxer.  This  was  the  most  abstract  of  the  stories 
and  the  one  which  required  the  most  concentration  on  the  part  of  the  viewer*  A  third 
narrative,  entitled  'Return  to  the  Sea',  was  about  a  beached  baby  whale.  Though  time  and 
nv  ice  were  Important  to  this  story*  the  primary  structure  of  the  narrative  was  a  problem  to 
be  resolved*  This  was  the  most  positive  of  the  four  narratives.  A  fourth  story,  entitled  'The 
Sound  of  Hatred'*  was  about  Apartheid.  It  was  written  to  be  an  emotion-evoking  depiction 
of  a  powerful  event:  the  attack  of  a  South  African  village  by  white  military.  In  this  narrative. 
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an  ominous  tone  was  set  at  the  beginning  of  the  story  when  the  narrator  asked:  'What  was 
that  sound?'.  The  subject  did  not  know  what  to  expect,  but  the  scenario  implied  something 
negative.  The  "whv"  of  the  event  was  never  given,  however  inferences  were  clearly 
presented  about  the  occurrence  to  suggest  racial  prejudice  and  Injustice.  It  should  be 
noted  that  three  of  the  stories  ('A  Child  is  Taken',  'Return  to  the  Sea'  and  'The  Sound  of 
Hatred')  were  more  schematically  predictable  than  the  other  storv  ('The  Strength  of 
Memories').  Also.  'A  Child  is  Taken'  and  'The  Sound  of  Hatred'  ended  on  a  very  negative 
emotional  tone,  whereas  The  Strength  of  Memories'  and  'Return  to  the  Sea'  ended  with 
positive  connotations  and  inferences  about  what  had  happened  and  was  expected  to 
hacpen  next. 


Procedure 

Since  each  storv  was  produced  In  fmu  metJIa  fotmat  conditions,  there  were  sixteen 
segments  to  be  manipulated  for  the  expat  iment.  F  our  sequences  were  made  and  each 
story  in  each  format  was  used  once.  The  01  <iei  of  the  format  was  manipulated  in  the  four 
sequences  of  the  experiment  to  that  each  foiinat  wan  in  eveiv  position  once.  No  format 
was  in  the  same  position  more  than  once:  howevel ,  a  story  may  have  been  In  a  position 
more  than  once.  The  four  presentation  sequences  used  In  the  experiment  are  liated  in 
Table  1. 
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Table  1  —  Order  of  Video  Presentation  for  Each  Sequence 


1             If  ] 

i    Ord  ;{                    Sequence  I  ! 

Sequence  II  J 

i~  1 

Miltifomat 

 1 

A  Child  is 
Taken 

Talking  Head 

Sound  of  Hatred 

t 

2 

V-U  Tfext 



Sound  of  Hatred 

ftiltifosnat 

Return  to  the 

Sea  ! 

3 

V-U  Video 



Return  to  the 
2ea 

V-U  Text 

Strength  of  { 
Menories  j 

4 

Talkie  tfead 

Strength  of 
Memories 

V-U  Video 

A  Oil  Id  is  j 
Taken  [ 

Orrl 

Sequence  III 

Sequence  IV  j 

1 

V-U  Video 

Sound  of  Hatred 

V-U  Text 

i 

Return  to  the 
Sea 

? 

Iklkinir  Huad 

Return  to  the 

Sea 

V-U  Video 

Strength  of 
Memories 

1 

Miltlfoiwt 

Strength  of 
Memories 

Talking  Head 

A  Quid  is 

Taken 

h 

V-U  Text 

A  Qiild  is 

Multi  format 

Sound  of  Hatred 

Hie  experiment  was  designed  to  shew  each  student  a  different  story  under  four 
diffract  media  font  of  presentation  conditions.  As  a  result*  each  format/story 
enmbi nation  ma  seen  bv  16  students.  Tbe  students  viewed  a  video  tape  of  the  four 
titnrieti,  completed  the  personal  media  information  sheet,  ranked  the  titles  of  the  stories 
ami  |x*r formed  four  tasks  for  each  story  —  verbal  recall,  spatial  task,  forcedrchoice 
rccoHnitiofi  and  continuation.  The  students  were  shown  the  tape  5ud  interview! 
Individually,   lie  total  time  to  view  tbe  tape  was  18  minutes.   Hie  total  tin  for  the 
experiment  was  1  hour  and  13  minutes.  The  dependent  variable  used  in  this  study  were: 
the  proportion  of  propositions  recalled  verbally:  tbe  proportion  of  forced  choice 
recognition  questions  answered  correctly;  the  proportion  of  propositions  recalled  in  Jie 
spatial  task:  and  the  type  of  elaborations  made  in  tbe  continuation  of  the  story.  As 
stated  previously,  this  paper  focuses  on  tbe  findings  of  tbe  verbal  recall  and  spatial 
task  activity. 

In  the  verbal  recall  analysis  tbe  interviews  asked  each  subject  tbe  following 
question  and  audio  taped  the  response  for  later  transcription  and  analysis:  Tbe  title  of 
the  first  story  you  saw  was:  (Title  of  Presentation)  Suppose  I  didn't  see  or  hear  this 
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presentation.   Tell  m  everything  you  can  remenber  about  this  story.**  There  were  four 
different  tasks  to  determine  spatial  recall.  Hie  task  for  "A  Qrild  is  Taken19  asked  the 
students  to  imagine  they  were  investigating  the  Armstrong  kidnapping  and  needed  a  nap  of 
the  scene  of  the  crime  which  would  show  the  route  taken  by  the  kidnappers  as  well  as  clues 
which  nay  have  been  left.   In  the  task  for  "The  Strength  of  Memories"  students  mere  asked 
to  imagine  they  wrote  history  books  about  boxers  and  wanted  to  include  a  sap  of  the  t  *m 
where  Salamo  was  raised.  Hie  spatial  task  for  "The  Sound  of  Hatred"  asked  subjects  to 
imagine  they  wer  -  newspaper  reporters  and  were  covering  the  raid  to  the  Crosstown 
Settlement.   They  needed  to  draw  a  sap  of  the  settlement  for  the  article.   In  "Return  to 
the  Sea"  students  were  asked  to  imagine  they  were  movie  directors  and  needed  to  draw  a 
story  board  of  the  scene  when  the  baby  whale  was  saved  by  the  villagers. 


Coding  Procedures 

Each  story  was  broken  down  by  propositions  (Kintsch  &  VanDyke.  1973)  for  analysis 
and  coding.   Subject  recall  of  the  narrative  was  coded  for  each  proposition  to  reflect  if 
the  proposition  was  recalled  and  whether  that  recall  was  an  exact  or  synonyoous 
replication  of  what  was  stated  in  the  text  or  whether  it  was  recalled  erroneously  by  the 
subject.   A  coding  sheet  was  also  designed  to  record  subject  recall  of  the  spatial  aspects 
of  the  presentation.  This  coding  sheet  served  as  a  way  to  evaluate  a  drawing  Bade  by  the 
subject  in  accordance  with  specific  propositions  from  the  presentation.  For  example  in 
The  Sound  of  Hatred"  certain  iteas  were  mentioned  in  the  narrative  such  as  shelters,  a 
road,  a  bus  stop  and  a  raw  sewage  ditch.  Hie  spatial  task  drawings  by  the  students  were 
evaluated  to  show  whether  the  students  drew  the  items,  textual! y. represented  the  items  on 
the  spatial  map  or  neglected  to  represent  the  iteas  in  any  manner  for  the  task. 


Results 
Overview 

The  inflrance  that  format  of  presentation  had  on  a  subject's  verbal  recall  of  a 
narrative  was  the  strongest  finding  in  this  study.  Mot  only  did  the  overall  verbal  recall 
of  the  narrative  differ  significantly  among  grows  of  students  who  saw  the  talking  head, 
voice-under-text,  voice-under-video  or  multifornat  presentation  coodition.  but  also,  when 
the  variance  due  to  the  influence  of  the  other  independent  variables  (story,  age  and 
gender)  was  rawed  in  an  analysis  of  covariance,  the  variance  among  the  different  format 
of  presentation  groups  continued  to  be  significant.  Furthermore,  whether  the  narratives 
were  analyzed  separately  or  together,  the  formats  that  generally  showed  the  highest  mean 
proportion  of  propositions  recalled,  were  the  primarily  visual  formats  (moltifozaat  and 
voice-under-video) .   Specifically,  the  proportion  of  propositions  verbally  recalled  by  the 
students  differed  significantly  among  the  four  formats  of  presentation:  students 
remembered  the  most  about  stories  shown  in  the  mm  visual  fonts  (multiforme  and  voice- 
aider-video) :  and  students  remenbered  the  least  from  the  primuily  audio  formats  (voice- 
mder-text  and  talking  bead) .  The  greatest  proportion  of  propositions  recalled  was  from 
narratives  shown  in  a  multifocal  presentation  and  the  least  proportion  of  propositions 
was  recalled  from  stories  seen  in  the  talking  hart  ormat.  Tim  spatial  task  findings  ware 
basically  inconclusive  and  will  be  discussed  la  • « 


c  .?0  m 


Verbal  Recall  Results 


Hie  proportion  of  propositions  recalled  verbally  were  comxtted  for  each  student 
according  to  the  coding  procedure  described  previously.  Analysis  of  this  data  showed 
significant  main  effects  for  three  (format .  story  and  ace)  of  the  four  independent 
variables  used  in  the  study.   Specifically,  the  proportion  of  propositions  verbally 
recalled  by  the  students  differed  significantly  among  the  presentation  f Gnats:  talking 
head,  voice-under-tcxt.  voice-under-video  and  aultif oimt  "(jF.(3.252)==11.29.jk.001). 
Furthermore,  when  the  nuisance  effects  of  the  other  independent  variables  (story,  age  and 
gender)  were  reooved  in  an  analysis  of  covariance.  the  proportion  of  propositions  verbally 
recalled  still  differed  significantly  aaong  the  talking  bead,  voice-under-text .  voice- 
under-video  and  ml ti format  presentation  formats  (£(3.249)=11.32.pc.001).  The  highest 
oean  proportion  of  propositions  recalled  was  by  students  who  saw  uultiformat  presentations 
(.29)  and  to  a  lesser  extent,  voice-under-video  presentations  (.27).  Students  who  saw  the 
stories  in  the  voice-under-text  (.23)  or  talking  bead  (.20)  formats  recalled  fewer 
propositions  fron  the  narratives. 

Table  2  shows  the  individual  and  overall  mean  and  standard  deviation  of  the 
proportion  of  propositions  recalled  frca  presentations  shown  as  a  talking  bead,  voice- 
under-tuxt.  voice-under-video  and  nultiforaat. 


Table  2  —  Proportion  of  Propositions  Recalled  Verbally 
 by  Format  of  Presentation  


• 

Talking 
Head 

V-u-Text 

V-u- 
Video 

i 

Multiformat  j  Entire  ] 
|  Population  j 

1 

1  Mean 
!    Std.  Dev. 

.1991 
.0764 

.2320 
.1032 

.2728 
.1076 

.2923        J  .2491  | 
.1070         j  .1052  j 

The  second  independent  variable  showing  a  strong  main  effect  significance  was 
story.  The  proportion  of  propositions  verbally  recalled  by  the  students  differed  significantly 
among  *A  Child  is  Taken*.  'Return  to  the  Sea*.  'The  Sound  of  Hatred*  and  'The  Strength 
of  Memories*  (F,(3.252)=5. 80,p=. 001 ).  in  general,  it  was  found  that  students  recalled 
the  most  from  the  two  positive  narratives.  'Return  to  the  Sea'  (.28)  and  'The  Strength  of 
Memories*  (.27).  The  most  predictable  story,  *A  Child  is  Taken*  (.23).  ranked  third  in  the 
proportion  of  propositions  verbally  recalled  by  students  in  the  study,  and  The  Sound  of 
Hatred*  (.22)  was  recalled  the  least  among  the  students.  This  last  story  most  resembles  a 
news  docu  mentary . 


Soatial  Task  Results 

The  proportion  of  propositionally  relevant  items  drawn  on  the  'spatial  map*  were 
analyzed  for  each  student.  Table  3  shows  the  mean  and  standard  deviation  of  that  recall 
by  format  of  presentation. 
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Table  3  — Praoortiono' 

f  Spatial  Items  Recalled  by  Format  of  Presentation 

Talking 
Head 

V-u- 
Text 

V-u- 
Video 

Multiform 

at 

Overall 

Mean 

Std.  Dev. 

.4008 
.1584 

.4206 
.1676 

.4134 
.1729 

.4272 
.1790 

.416 

Analysis  of  variance  of  the  proportion  of  Hems  recalled  by  students  during  the 
soatial  task  did  not  show  independent  variable  effects  for  format,  age  or  gender.  However, 
the  proportion  of  spatially  recalled  items  did  ihow  a  main  effect  story  significance  among 
'A  Child  is  Taken',  'Return  to  the  Sea',  'The  Sound  of  Hatred*  and  The  Strength  of 
Memories'  (fl3,224)»21 .50.  p<  .001 ).  Students  recalled  the  most  from  'Return  to  the 
Sea'  (.52)  and  progressively  less  from  The  Strength  of  Memories'  (.44)  ?  'A  Child  is 
Taken'  {.  39)  and  finally  'The  Sound  of  Hatred'  (.32).  This  was  consistent  with  the  verbal 
recall  findings  in  the  study.  The  two-way  Interaction  of  format  and  story  was  also 
significant  (R9.20QW2.06,  p<  .05),  but  none  of  the  other  interactions  were  significant. 

Since  story  was  the  only  independent  variable  that  showed  a  main  influence  on 
recall  for  spatial  task,  the  stories  were  investigated  further  to  see  if  there  was  an  individual 
story-format  relationship.  Table  4  shows  the  mean  proportion  of  spatial  items  recalled 
when  the  stories  were  analyzed  separately.  Only  The  Sound  of  Hatred' differed 
significantly  in  the  proportion  of  items  recalled  by  students  who  saw  the  voice-under-text 
(.40),  talking  head  (.35),  voice-under-video  (.26)  or  muttiformat  (.26)  presentations 
(/=T3?60)  =  3.75f  p<  .05). 


Table  4  —  Mean  Proportion  of  Spatial  Items  Recalled 
  by  Format  and  Storv 


Talkina 
Head 

V-u- 
Text 

V-u- 
Video 

Multiformat 

Total 

Population 

CHILD 

.35 

.34 

.39 

.46 

.39 

RETURN 

.50 

.51 

.52 

.55 

.52 

SOUND 

.35 

.40 

.26 

.26 

.32 

STRENGTH 

.40 

.44 

.48 

.44 

.44 

Conclusions 

The  findings  reported  here  suggest  that  format  of  presentation  does  influence  the 
memory  for  a  narrative.  Significant  differences  in  the  proportion  of  propositions  verbally 
recalled  were  found  among  stories  presented  as  talking  head,  voice-under-text,  voice- 
under-video  and  multiformat  presentations.  Students  who  viewed  the  primarily  visual 
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formats  (muftiformat  and  voice-under-video)  showed  the  highest  recall  and  students  who 
viewed  the  primarily  audio  formats  (voice-under-text  and  talking  head)  recalled  fewer 
propositions.  Also,  more  was  recalled  from  the  more  positive  narratives  than  the  stories 
with  a  relatively  negative  slant. 

Since  the  verbal  recall  findings  were  so  strong,  it  showed  that  there  were  memory 
differences  associated  with  presentation  formats,  but  the  coding  procedures  used  to 
evaluate  the  spatial  tasks  of  the  students  were  so  rough  they  were  unable  to  adequately 
reflect  the  differences  in  the  drawings.  The  coder  was  instructed  to  look  for  certain  items 
on  the  spatial  task  sheet  and  mark  only  whether  they  were  represented  in  some  manner  by 
the  student.  As  a  result,  this  static  evaluation  lost  the  creativity  of  the  drawings  as  well  as 
their  uniqueness.  For  example,  as  can  be  seen  in  the  attached  drawings,  subjects  4  and 
46  saw  the  same  version  of  the  experiment.  However,  each  subject  had  a  different 
interpretation  of  the  tasks.  Some  drawings  are  very  'busy*  and  the  student  completely 
covered  the  space  provided  for  the  spatial  task  with  incidental  scenery,  people  or  other 
information  about  the  story.  Another  drawing  by  that  same  student  is  a  very  verbal,  textual 
representation  the  task  for  the  story.  This  contrast  in  drawings  by  the  same  student  was 
neither  recognized  nor  evaluated  with  the  current  coding  scheme.  More  sensitive  coding 
procedures  could  be  developed  to  identify  unique  drawing  patterns  within  each  format  of 
presentation  for  comparison  and  analysis. 

Furthermore,  the  fact  that  the  task  for  each  story  was  different  confounded  the 
problem  of  accurately  interpreting  a  student's  spatial  memory  for  a  story.  For  example,  the 
most  structured  task  was  for  'Return  to  the  Sea*  and  included  a  story  board  for  students 
to  use  to  draw  certain  scenes  from  the  narrative.  Students  were  able  to  draw  the  most 
oropositionally  relevant  items  for  this  story.  On  the  other  hand,  the  tasks  associated  with 
the  other  three  stories  gave  students  about  the  same  amount  of  structure  to  stimulate  their 
response  in  the  spatial  drawing.  It  is  interesting  that  the  proportion  of  spatial  items  recalled 
from  these  stories  were  about  the  same. 

Finally,  a  coding  scheme  could  be  developed  to  see  if  there  was  a  positive-verbal 
and  negative-visual  relationship  among  the  presentation  formats,  the  stories,  and  the 
drawings.  For  example,  did  students  who  saw  the  primarily  visual  presentation  formats 
(voice-under-video  and  multif ormat)  display  more  recall  in  the  spatial  task  for  the  more 
negative  stories  ('The  Sound  of  Hatred*  and  *A  Child  is  Taken*)  than  the  positive  stories? 
Similarly,  did  students  who  viewed  the  primarily  audio  presentation  formats  (talking  head 
and  voice-under-text)  display  less  recall  of  the  negative  stories  than  the  more  positive 
stories  (*Return  to  the  Sea*  and  'Strength  of  Memories*)?  The  coding  procedure  to 
determine  this  relationship  would  need  to  reflect  a  sensitivity  to  the  media  formats,  the 
emotional  valence  of  the  stories,  the  spatial  task  of  the  drawing  and  individual  differences 
in  a  subject's  drawing  ability.  That  information  could  also  then  be  compared  to  the 
students  verbal  recall  and  forced  recognition  performance  of  this  study. 

Much  more  analysis  is  needed  to  further  advance  the  implications  of  the  findings  in 
this  particular  portion  of  the  study.  However,  if  one  conclusion  can  be  drawn  from  the  data 
provided  in  this  study,  it  is  that  each  format  of  media  presentation  has  unique  attributes 
which  interact  with,  and  enhance  the  memory  for,  a  variety  of  information. 
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Introduction 

Walter  Dick  has  reminded  us  that  with  the  advent  of  new  ideas,  such  as  constructivism  and 
total  quality  management  in  education,  we  are  faced  with  the  need  to  reassess  the  viability  of  our 
existing  paradigms  and  models,  such  as  instructional  systems  design  (Dick,  1993).  Models,  such  as  the 
systems  approach,  remain  viable  as  they  are  able  to  be  used  to  solve  our  problems.  Although  ISD  has 
traditionally  been  used  as  a  micro-level  model  to  design  instructional  interventions,  there  hai  been,  in 
recent  years,  an  encouraging  trend  of  using  the  systems  approach  for  more  macro-level  applications, 
such  as  performance  technology  (Mager,  1987;  Mager  &  Pipe,  1984;  Rossett,  1990)  and  educational 
systems  design  (Banathy  &  Jenks,  1990;  Kahn  &  Reigeluth,  1993;  Salisbury,  1990). 

The  purpose  of  this  paper  is  fourfold: 

•  To  provide  an  overview  of  nontraditional  degree  programs. 

•  To  demonstrate  that  the  difference  between  traditional  instruction  and  ISD  is  analogous  to  the 
difference  between  traditional  and  nontraditional  degree  programs. 

•  To  present  an  example  of  how  a  popular  instructional  systems  design  model  can  be  adapted  for 
use  at  a  macro  levol  to  design  nontraditional  degree  programs  for  adult  students. 

•  To  demonstrate  that  the  systems  approach  is  a  viable  model  that  can  offer  solutions  to  various 
types  of  educational  problems. 

The  problem 

Adult  re-entry  students,  those  25  years  of  age  or  older  who  have  returned  to  pursue  college 
studies,  are  the  fastest  growing  segment  of  our  student  population.  Of  all  currently  enrolled  college 
students,  two  out  of  five  belong  to  this  group  (Klein,  1990).  Until  recently,  our  present  college  system 
has  had  difficulty  meeting  the  needs  of  these  learners,  who  often  feel  out  of  place  or  "behind"  younger 
peers  who  entered  college  straight  from  high  school  (Klein,  1990;  Pierson  and  Springer,  1988).  Re- 
entry students  are  generally  highly  motivated  (Pierson  and  Springer,  1988)  and  achieve  at  a  level  equal 
to  or  higher  than  younger  students  (Klein,  1990).  It  appears  that  adult  re-entry  students  bring  certain 
critical  skills  and  experience  that  enables  them  to  more  fully  avail  themselves  of  educational 
opportunities. 

Unfortunately,  many  professionals  and  other  working  adults  have  encountered  a  great  deal  of 
difficulty  in  their  efforts  to  further  their  education.  Many  factors  contribute  to  this  difficulty,  including 
a  lack  of  available  courses  that  meet  after  work,  location  of  classes,  insufficient  independent  study  or 
correspondence  courses,  difficulty  in  transferring  units  from  several  institutions,  residency 
requirements,  and  lack  of  programs  that  meet  individual  learner  needs.  Many  of  these  adult  learners 
are  facing  a  dilemma.  They  find  themselves  in  need  of  a  college  degree  to  advance  in  their  careers  or 
desire  personal  enrichment,  but  cannot  leave  work  to  do  so. 

Although  professionals  often  have  skills  and  expertise  which  far  surpass  lesser  experienced  full 
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time  college  nIiuIhjiIn,  limy  tiwy  lark  MiHIrmnt  'iniiu*  lor  an  undergraduate  degree.  Many  express  a 
feeling  of  frujitialloti  (la.  'Willi  my  cuiietit  uhedule*  Til  uavar  have  time  to  finish  my  degree.") 

Special  programs  for  tuiull  Itimutrs 

In  response  In  thine  ruwda,  many  colleen  and  universities  are  now  providing  special  programs 
that  allow  working  adulta  to  complete  thair  degreea  by  attending  claaaea  at  nights  or  on  weekends, 
sometimes  at  their  actual  work  lite.  Many  of  these  institutions  offer  accelerated  courses  that  can  be 
completed  in  less  than  a  typical  semester.  Mora  Institution!  of  higher  learning  are  offering  independent 
study  and  correspondence  couraea  and  accepting  higher  amounts  of  these  courses  as  transfer  units.  The 
amount  of  courses  taught  via  distance  learning,  television,  videotapes,  cassettes  and  other  forms  of 
alternative  delivery  increase  dramatically  with  each  new  year. 

Several  colleges  have  established  learning  centers  in  cities  and  military  bases  across  the  nation. 
Students  may  pursue  degrees  at  these  centers  and  complete  their  residency  requirements  without  having 
to  take  courses  at  the  home  campus. 

Nontraditioninal  degree  programs 

More  innovative  and  controversial  than  the  external  academic  centers  are  the  "alternative"  or 
"nontraditional"  external  programs.  These  programs  have  characteristics  that  differ  radically  from 
traditional  degree  programs  (Bear,  1988,  1990;  Thorson,  1992). 

•  They  often  require  little  or  no  residency  at  the  degree  granting  institution 

•  Nontraditional  programs  are  built  upon  the  concept  that  college-level  knowledge  can  be  learned 
both  inside  and  outside  the  classroom.  These  programs  tend  to  be  much  more  liberal  about 
how  degree  requirements  may  be  fulfilled  than  traditional  programs. 

•  Knowledge  gained  outside  the  classroom  can  be  assessed  by  College  Level  Examination 
Programs  (CLEP),  Graduate  Record  Examinations  (GRE),  D ANTES  examinations  (military), 
and  portfolio  assessments.  These  assessments  allow  learners  with  college  level  knowledge  to 
receive  credit  for  their  skills  and  expertise  and  reduce  the  length  of  time  to  finish  their 
degrees. 

•  Alternative  learning  experiences,  such  as  books  written,  research  reports,  computer  programs, 
and  other  independent  projects  can  be  used  to  fulfill  the  requirements  for  a  degree. 

•  Nontraditional  programs  generally  assume  that  the  learner  has  amassed  a  high  degree  of 
knowledge  and  experience  prior  to  commencing  his  or  her  degree  program. 

•  Many  programs  allow  individualized  majors  to  be  designed  by  the  learner,  in  conjunction  with 
a  faculty  mentor  or  committee. 

Differences  between  traditional  and  nontraditional  degree  programs  are  illustrated  in  Table  1. 
A  few  of  the  tenets  of  alternative  programs,  such  as  the  awarding  of  college  credit  for  experiential 
learning*  have  met  with  criticism  (Bailey,  1979).  Some  educators  have  expressed  concern  about  the 
quality  of  nontraditional  programs  (Bowen,  Edlestein,  and  Medaker,  1979),  while  others  think  that  any 
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program  that  deviates  from  the  well-established  norm  must  be  a  diploma  mill. 

Quality  of  nontraditional  degree  programs 

Advocates  of  alternative  degree  programs  point  to  the  fact  that  their  institutions  are  accredited 
by  the  same  regional  associations  that  accredit  the  traditional  schools  (Andrews,  1973)  and  that  the 
American  Council  on  Education  has  clear  guidelines  ferine  evaluation  of  such  programs  (ACE  Task 
Force,  1990).  Another  testament  is  the  fact  that  one  of  the  most  popular  guides  of  alternative  programs 
lists  well  over  100  accredited  colleges  and  universities  that  offer  off-campus  college  degrees,  including 
some  very  well-known  and  prestigious  private  and  state  universities  (Thorson,  1992).  It  is  claimed  that 
these  programs  are  geared  toward  individual  learner  needs,  rather  than  institutional  needs,  and  that' 
greater  involvement  and  active  learning  is  required  from  each  student. 

Studies  conducted  in  Michigan  and  Texas  suggest  that  degrees  achieved  through  alternative 
programs  are  successful  in  meeting  the  needs  of  adult  learners  who  enroll  in  these  programs.  In  the 
Michigan  study,  130  traditional  and  182  nontraditional  alumni  showed  no  significant  differences  in  their 
perception  of  their  degree's  importance  in  improving  pay  and  promotion,  effectiveness  in  obtaining 
jobs,  ability  to  improve  job  skills  and  work  performance,  and  enhancement  of  career  development 
(Firenze,  1984). 

Nontraditional  students  rated  significantly  higher  in  satisfaction  of  having  the  degree, 
enjoyment  of  the  learning  experience,  and  desire  to  obtain  prerequisites  for  graduate  education  than 
their  traditional  counterparts.  Nontraditional  students  also  placed  a  higher  value  on  learning  the  subject 
area  for  pure  interest  purposes  (Fircnze,  1984). 

In  the  Texas  study,  863  graduates  of  a  nontraditional  program  showed  a  high  degree  of 
satisfaction  in  career  usefulness  of  the  degree,  improved  self  image,  level  of  difficulty  compared  to 
traditional  programs,  and  salary  increase.  It  was  suggested  that  the  quality  of  both  traditional  and 
nontraditional  degree  programs  should  be  assessed  by  the  satisfaction  of  its  graduates,  rather  than  by 
less  relevant  criteria,  such  as  student  selectivity,  admissions  and  programmatic  homogeneity  (Pierson 
and  Springer,  1988). 


Systematic  instructional  design  as  a  metaphor 

Proponents  of  instruction  systems  design  (Dick  and  Carey,  1990;  Gagnl,  Briggs  and  Wager, 
1988;  Knirk  and  Gustafson,  1986)  have  held  to  a  more  learner-centered  conception  of  the  instructional 
process  (Reiser,  1987).  When  instruction  is  developed  according  to  a  systems  view,  it  is  viewed  in 
terms  of  learner  performance,  rather  than  teacher  or  institutional  performance  (Mager,  1987).  This 
learner-centered  orientation  of  instructional  designers  is  also  the  orientation  championed  by  proponents 
of  alternative  degree  programs. 

The  literature  on  alternative  degree  programs  versus  traditional  degree  programs  is  scanty. 
However,  the  differences  between  systematic  instructional  development  and  traditional  institution- 
centered  instruction  are  well  documented.  A  recent  ERIC  search  produced  no  models  for  alternative 
degree  programs.  Several  different  models  for  instructional  development,  on  the  other  hand  are  present 
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in  the  literature. 

Traditicnal  institutional  model  of  instruction 

The  flow  diagram  in  Figure  1  illustrates  how  instruction  has  been  handled  traditionally  under 
the  institution  model.  This  is  the  model  most  commonly  encountered  in  our  present  educational 
system. 

First,  the  curriculum  or  content  of  an  instructional  program  is  formulated.  In  the  traditional 
model,  this  is  accomplished  at  the  administrative  level.  School  boards  or  adr  inistrators  determine  what 
is  necessary  to  meet  learner  needs. 

Once  the  content  has  been  established,  it  is  then  divided  into  manageable  units  or  lessons, 
either  by  the  institution  in  its  che  ce  of  texts,  or  by  the  teacher.  Thg  content  is  then  taught  to  the 
learners.  The  major  objective  of  instruction  in  the  traditional  system  is  the  teaching  of  the  unit  or 
lesson  content  to  the  learners.  This  is  most  often  accomplished  through  lecture. 

After  the  content  has  been  disseminated,  the  teacher  administers  an  examination  based  on  the 
content.  Following  the  tallying  of  the  test  scores,  a  norm  referenced  (bell  curve)  scale  is  most  often 
utilized  to  compare  a  given  learner's  score  with  the  others  in  the  class.  Passing  the  course  is  achieved 
when  a  certain  number  of  units  or  lessons  are  completed  at  a  specified  level  (ie.  "C  grade  or  above). 

In  the  traditional  institutional  model  of  instruction,  learners  play  a  relatively  passive  role. 
They  have  little  input  in  deciding  what  they  need  to  learn  and  how  they  are  to  leana  U,  The  rigid 
nature  of  the  institutional  model  makes  it  difficult,  if  not  impossible,  to  tailor  tue  i^oguut  *o  individual 
differences  and  needs  of  learners. 

Traditional  institutional  model  for  degree  programs 

The  flow  diagram  in  Figure  2  illustrates  the  process  in  formulating  and  awarding  a  traditional 
college  degree.  The  similarities  to  the  traditional  instructional  model  are  apparent: 

•  The  Institution's  administration  decides  upon  the  general  education  and  specific  major 
curriculum. 

•  The  instructional  content  is  divided  into  specific  courses  offered  by  the  institution. 

•  Course  material  is  taught  to  the  learner. 

•  Acceptable  passing  level  is  determined  for  the  courses*  (ie*  "C  grade  or  above) 

•  A  degree  is  awarded  when  a  certain  number  of  prescribed  courses  are  completed  with  a  "C" 
grade  or  above. 

As  with  the  traditional  mode,  of  instruction,  the  traditional  degree  program  is  designed  and 
administered  from  an  institutional  point  of  view.  Learners  are  processed  through  the  system,  but  have 
little  voice  in  the  development  of  the  system. 
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Systematic  instructional  design 

The  instructional  designer  who  utilizes  a  systems  approach  must  address  three  important 

issues: 

•  What  should  the  learner  know  and  be  ante  to  do  at  the  completion  of  the  instruction? 
Answers  to  this  question  allow  .ue  developer  to  formulate  the  instructional  objectives  and 
develop  the  evaluation  instruments  for  the  objectives. 

•  Which  of  these  objectives  does  the  learner  already  know?  Answering  this  question  helps 
the  developer  to  eliminate  redundancy  in  the  instruction  and  lets  the  teacher  know  what  jjoj  to 
teach. 

•  What  is  the  most  effective  and  efficient  way  to  help  the  learner  meet  the  objectives?  The 
answers  to  this  question  determine  the  content  of  the  instruction  and  allows  the  learner  to 
receive  instruction  with  maximum  efficiency  and  minimum  waste. 

Systematic  alternative  degree  design 

The  developer  of  an  alternative  external  degree  program  utilizes  the  same  learner  centered 
thinking: 

•  What  should  the  learner  kraw  and  be  able  to  do  to  qualify  for  a  bachelor  of  arts  degree? 
A  body  of  competencies,  rather  than  a  specific  sequeace  of  courses,  become  the  educational 
objectives. 

•  Which  of  these  objectives  docs  the  learner  already  know?  An  evaluation  of  prior  learning 
decides  how  many  of  these  objectives  have  already  been  met  by  the  learner. 

•  What  is  the  most  effective  and  efficient  way  to  help  the  learner  meet  the  objectives? 
Appropriate  instructional  strategies  may  be  utilized  to  give  the  learner  the  remaining 
competencies  for  the  degree. 

Toward  a  model  for  development  of  alternative  degree  programs 

Figure  3  is  an  illustration  of  one  of  the  most  widely  utilized  systems  mode:  ji  instructional 
design  (Dick  and  Carey,  1990;  Gagnl,  Briggs  and  Wager,  1988).  This  model  can  easily  be  adapted  for 
the  designer  of  an  alternative  degree  program.  Figure  4  illustrates  this  adaptation. 

Both  instructional  and  alternative  program  designers  must  identify  the  goals  of  their  programs 
fie.  what  is  to  be  accomplished?).  Needs  and  task  analyses,  commonly  utilifcdd  by  instructional 
designers,  cam  be  used  also  by  the  degree  program  designer  to  determine  wf.it  the  competency 
requirements  of  the  degree  should  be.  The  instructional  designer  and  the  degree  program  designer  need 
to  identify  prior  knowledge  in  order  to  avoid  teaching  what  the  learner  already  knows,  thus  increasing 
efficiency  and  decreasing  redundancy*  Subtracting  the  learner's  prior  knowledge  ftxm  the  competency 
requirements  leaves  the  objectives,  which  form  the  basis  for  the  program* 
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Systematic  instructional  design  requires  criterion  referenced  (mastery  level),  rather  than  norm 
referenced  measures.  Criterion  referenced  measures  are  based  on  the  program's  objectives.  Learners 
in  an  alternative  degree  program  should  advance  only  when  they  have  mastered  all  of  a  given  course's 
objectives,  not  when  they  have  received  a  70  percent  score  on  a  rote  memory  test.  The  designers  must 
then  decide  upon  the  strategies  (attending  classes,  distance  learning,  independent  study,  research 
projects,  etc.)  that  will  best  help  the  learner  achieve  the  program's  objectives. 

Once  the  program  requirements  and  structure  have  been  designed,  appropriate  experiences 
must  be  developed  or  selected  for  use  by  the  learner.  The  effect  of  the  program  is  then  evaluated  and, 
if  needed,  revised. 

Discussion 

Robert  Mager,  Charles  Reigeluth,  Allison  Rossett,  David  Salisbury  and  others  are  encouraging 
the  rest  of  us  ISD'ers  to  look  beyond  designing  instruction  and  use  our  expertise  to  solve  corporate  and 
institutional  problems  (Dick,  1993).  One  such  problem  that  can  benefit  from  a  systems  approach  is  the 
design  of  learner-sensitive  programs  for  adult  students. 

The  application  of  a  systems  approach  to  degree  program  planning  will  require  a  paradigm 
shift  (Kuhn,  ),  just  as  ISD  requires  a  change  from  the  traditional  model  of  instruction.  Well-establish 
concepts,  such  as  the  Carnegie  Unit,  will  need  to  be  reassessed  along  with  our  other  models,  to  see  if 
they  are  still  viable. 

The  population  of  adult  re-entry  students  will  likely  increase,  rather  than  decrease,  in  the 
coming  years.  As  college  degrees  become  required  credentials  for  More  professions,  the  issue  of 
educational  access  for  adult  learners  will  continue  to  rise  in  importance.  Higher  education  programs 
will  need  to  accommodate  those  who  study  while  they  maintain  full  time  jobs.  Nontraditional 
alternative  degree  programs  may  provide  a  viable  option  for  these  learners.  According  to  graduates  of 
alternative  programs,  their  degrees  have  met  their  needs  and  have  been  as  useful  as  degrees  earned 
through  traditional  institutional  programs.  Models  and  procedures  utilized  to  design  learner  based, 
mastery  level  instruction  are  the  same  models  which  can  be  utilized  by  those  designing  learner  based 
degree  programs. 
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Table  1:     Some  Differences  Between  Traditional  and  Nontraditional 
Degree  Programs  (After  Thorson,  1992) 


Traditional  Nontraditional 


Provides  regimented  dependence 
on  traditional  systems. 

Classroom  is  main  source  for 
exchange  of  knowledge. 

Curricula  oriented  to  school's 
tradition  and  societal  needs. 

Faculty  member  transmits  block 
of  knowledge  to  learners. 


Degree  requirements  based  on 
bureaucratic  standards. 


Degree  awarded  when  all 
prescribed  classes  have  been 
completed. 


Provides  self-direction  and 
autonomy. 

Independent  self-discovery  is 
main  source  of  knowledge. 

Curricula  oriented  to  learner's 
personal  and  professional  needs. 

Faculty  member  acts  as  mentor 
and  facilitate  •:  of  learning 
experiences. 

Degree  requirements  based  on 
learner's  needs,  goals  and  on 
practical  standards. 

Degree  awarded  by  credits  based 
on  previous  college-level 
knowledge  anc  academic 
standards  agreed  to  by  faculty 
and  learner. 
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Figure  1 : 

Institutional  Model 
of  Instruction 

Figure  2: 

Institutional  Model 
for  Degree  Programs 
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Figure  3: 
Systematic  Process  of 
Instructional  Design 

(After  dick  &  Carey,  1990) 
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Figure  4: 

Proposed  model  for  the 

Design  of  Nontraditional 
Degree  Programs 
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Visualization  as  an  Aid  to  Problem-Solving:  Examples  from  History 


Lloyd  P.  Rieber 
The  University  of  Georgia 

This  paper  presents  an  historical  overview  of  visualization  as  a  human  problem-solving  tool.  Visualization 
strategies,  such  as  mental  imagery,  pervade  historical  accounts  of  scientific  discovery  and  invention.  A  selected 
number  of  historical  examples  are  presented  and  discussed  on  a  wide  range  of  topics  such  as  physics,  aviation, 
and  the  science  of  chaos.  Everyday  examples  are  also  discussed  to  show  the  value  of  visualization  as  a  problem* 
solvingtoolforall  people.  Several  counter  examples  are  alsodiscussed  showing  that  visualization  can  sometimes 
lead  to  erroneous  conclusions.  Many  educational  implications  are  discussed,  such  as  reconsidering  the  dominant 
role  and  value  schools  place  on  verbal,  abstract  thinking.  These  issues  are  also  considered  in  light  of  emerging 
computer-based  technologies,  such  as  virtual  reality. 


The  increased  availability  of  multimedia  tools  in  educa- 
tion permit  the  design  of  instructional  systems  that  incorporate 
unlimited  variations  and  forms  of  textual,  visual,  aural  infor- 
mation for  both  presentation  and  feedback.  However,  our 
sense  of  vision  arguably  represents  our  most  diverse  source  of 
information  of  the  world  around  us  (Sekular  &  Blake.  1985). 
Society,  including  education,  transmits  tremendous  amounts 
of  information  in  visual  form.  Visualization  is  most  frequently 
used  in  instruction  in  the  presentation  of  information.  How- 
ever, visualization  techniques  are  also  powerful  problem- 
solving  tools,  though  they  are  rarely  promoted  as  such  in 
learning  and  instruction.  This  is  unfortunate,  as  history  is  full 
of  fascinating  examples  where  visualization  has  been  one  of 
our  most  important  arsenals  of  problem-solving  tools 
(Koestler.  1964). 

The  purpose  of  this  paper  is  provide  ahistorical  context  for 
current  efforts  by  instructional  technologists  to  exploit  the  full 
potential  of  visualization  techniques,  especially  those  that  tap 
the  computational  anil  graphical  power  of  the  computer.  This 
paper  will  present  a  variety  of  cases  where  people  have  used 
visualization  techniques  throughout  history  to  solve  a  wide 
range  of  problems.  The  term  visualization  is  used  broadly  here 
to  include  all  nonvcrhal  cognitive  strategies,  including  mental 
image ry.  tn  addition,  some  counter  examples  will  be  discussed 
in  show  \ww  visualization  can  sometimes  lead  us  astray.  The 
paper  will  conclude  by  discussing  the  implications  of  visual- 
ization in  education  and  instructional  design.  These  issues  will 
surely  increase  in  importance  and  complexity  as  highly- visual 
computer -based  systems  continue  toevolvc,  such  as  in  the  case 
nf  vii  tiidl  reality.  This  paper  suggests  that  history  might  be  able 
to  help  us  as  we  struggle  for  the  most  appropriate  applications 
of  visualization  in  education  both  now  and  in  the  future.  This 
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paper  begins  with  a  few  general  examples  that  illustrate  the  role 
of  visualization  in  human  problem-solving  at  the  everyday 
level. 

Visualization  as  a  General  Problem-Solving 
Strategy 

Although  historical  examples  of  famous  people  using 
visualization  to  solve  complex  problems  are  often  the  most 
dramatic.everyday  people  using  the  same  visual  skills  to  solve 
everyday  problems  are  the  most  poignant.  Some  problems,  of 
course,  are  inherently  spatial.  Consider  giving  or  getting 
directions  to  an  unfamiliar  pan  of  town.  It  is  interesting  how 
often  the  "direction  giver"  usually  starts  with  a  pure  verbal 
description,  but  then  quickly  reverts  to  visualization  "tricks" 
extemporaneously  (such  as  pointing  in  the  air  to  illustrate  the 
many  turns  and  distances).  It  is  almost  as  if  the  person  is  going 
on  a  brief,  imaginary  trip  to  the  final  destination  on  behalf  of 
the  lost  individual  in  the  hope  to  show,  by  example*  how  to  get 
theTC.  The  "direction  getter"  is  at  the  same  time  trying  to 
mentally  form  a  visual  imprint  of  the  trip  while  memorizing 
key  verbal  labels  (such  as  landmarks  and  street  names). 

Researchers  who  study  the  problem-solving  process  have 
long  recognized  visualization  as  a  problem-solving  tool 
(Finke.  1990;  Ftnke,  Ward.  Smith.  1992).  People  often  times 
forget  to  use  such  inherent  capabilities,  perhaps  because 
schools  tend  to  emphasize  verbal  skills  over  visual  skills  and 
abstract  reasoning  over  concrete  reasoning.  Unfortunately ,  the 
idea  of  using  simple  visualization  as  a  cognitive  strategy  to 
help  ourselves  solve  a  problem  isoftcn  times  cither  overlooked 
or  discouraged.  For  example,  consider  the  following  problem 
(from  Branaford  &  Stein,  1984): 

A  man  had  four  chains,  each  three  links  long.  He  wanted  to 
join  the  four  chains  into  a  single,  closed  chain.  Having  a  link 
opened  cost  2  cents  and  having  a  link  closed  cost  3  cents. 
The  man  had  his  chains  joined  into  a  closed  chain  for  \5 
cents.  How  did  he  do  it? 

Take  a  few  mrenenis  to  try  to  solve  the  problem  before 
reading  ahead  for  the  solution.  As  you  do  so.  reflect  on  the 
strategies  that  you  are  using  to  solve  the  problem. 
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Most  people  find  the  problem  very  difficult  to  solve 
mentally .  The  first  possible  solution  of  four  links  opened  and 
closed  would  cost  twenty  cents.  For  most  people,  a  good  first 
step  is  to  draw  the  four  chains  on  paper — to  construct  a  visual 
representation  of  the  problem's  entry  conditions.  After  work- 
ing through  opening  and  closing  links  with  a  visual  model,  one 
discovers  that  the  solution  rests  in  opening  all  three  linksofone 
of  the  chains.  These  three  links  can  then  be  used  to  join  the 
other  three  chains.  When  the  problem  is  converted  into  visual 
form,  the  solution  is  easy  to  derive. 

Lave  (1988)  has  described  the  ways  everyday  people 
solve  problems  by  exploiting  all  of  the  resources  present  in  the 
problem  situation.  For  example,  people  were  presented  with 
an  everyday  problem  of  fixing  food  for  three  people  when  the 
recipe  listed  amounts  for  four  people.  One  recipe  called  for  two 
thirds  cup  of  cottage  cheese.  One  person  quickly  solved  the 
abstract  problem  of  finding  "three  fourths  of  two  thirds"  by  first 
measuring  out  two  thirds  cups  onto  the  table,  patting  it  into  a 
circle,  and  marking  a  cross  on  it.  The  person  then  removed  the 
one  excess  quarter  and  was  left  with  the  correct  portion! 
Lave's  work  is  often  cited  by  proponents  of  situated  cognition 
(e.g.  Brown,  Collins,  ADuguid.  1989),  though  these  examples 
also  show  how  everyday  people  use  spatial  and  concrete 
reasoning  abilities  to  grapple  with  problems  often  expressed  in 
abstract  form  in  traditional  mathematics.  The  concrete  solu- 
tions are  just  as  sophisticated  and  complex  as  those  expressed 
abstractly,  yet  such  a  visualization  strategy  would  probably  not 
be  allowed  by  most  math  teachers. 

Here  is  another  example  that  aptiy  demonstrates  the  power 
of  visualization  in  problem-solving  (Norman,  1988).  Even 
more  so.  it  reveals  the  power  of  the  human  perceptual  system 
to  deal  with  problems  efficiently  and  effectively  when  pre- 
sented in  visual  form.  The  problem  is  a  simple  math  strategy 
game  for  two  players.  The  game  starts  by  writing  the  numbers 
one  to  nine  on  index  cards  —  one  number  per  card.  The  nine 
cards  are  laid  out  on  a  table  with  the  numbers  facing  up.  The 
players  then  take  turns  choosing  a  number.  Each  number  can 
only  be  chosen  once  in  a  game.  The  first  player  who  gets  any 
combination  of  three  numbers  that  add  up  to  15  is  the  winner. 
The  game  is  quite  a  challenge  even  for  adults.  One  must 
inticipare  appropriate  combinations  of  three  numbers  sum- 
ming to  1  5,  while  also  anticipating  possible  winning  combina- 
tions by  their  opponent.  Numbers  are  chosen  either  to  advance 
one's  own  hand  or  to  block  an  approaching  win  by  the  oppo- 
nent. Try  playing  this  game  a  few  times  with  a  friend  before 
reading  further. 

Playing  this  game  in  the  pure  mathematical  form  de- 
scribed above  is  quite  difficult.  Many  adults  do  not  remember 
ever  playing  this  game,  though  most  find  it  strangely  familiar. 
The  reason  for  this  is  that  the  game  has  another,  more  familiar 
form  —  tic-tac-toe.  The  commonalty  of  the  games  can  be 
recognized  by  carefully  arranging  the  nine  numbers  so  that  all 
vertical,  horizontal,  and  diagonal  combinations  of  three 
squares  add  up  to  15.  as  illustrated  in  Figure  1  (this  special 
combination  is  also  known  as  a  "magic  square.")  Strategies 
from  one  version  of  the  game  quickly  transfer  to  the  other  For 
example,  capturing  the  middle  square  or  the  number  "5"  gives 
the  player  a  distinctive  advantage.  Most  adults  consider  tic- 
tac-toe  to  be  a  simple  child's  game  not  worth  playing  anymore 
because  the  game  will  inevitably  end  up  in  a  draw  once  both 


players  understand  the  "secret"  to  successfully  blocking  the 
opponent  at  every  move.  Interestingly,  the  pure  math  version 
of  the  game  remains  a  challenge  even  knowing  that  it  is  a 
"disguised"  version  of  tic-tac-toe.  The  point  is  that  the  game 
becomes  "childish"  only  when  the  perceptual  ability  of  pattern 
recognition  is  used  The  game  itself  has  not  changed,  only  the 
cognitive  tools  used  by  the  individual  to  play  it. 

Visualization  by  Scientists  and  Inventors 

Someof  the  most  fascinating  accounts  of  human  problem- 
solving  show  remarkably  simple  examples  of  how  visualiza- 
tion coupled  with  imagination  led  to  brilliant  discoveries  and 
flashes  of  insight  (Burke,  1985;  Shepard,  1988).  It  is  interest- 
ing that  we  often  refer  to  these  people  as  "visionaries.**  some- 
how being  able  to  see  what  others  cannot  The  use  of  this 
description  may  not  be  as  metaphorical  asone  might  first  think. 
It  is  stunning  how  many  scientists  and  inventors  placed  a  great 
deal  of  importance  on  the  nonverbal  in  the  act  of  creative 
imagination.  Many  describe  the  phenomena  of  sudden  "illu- 
mination" where  solutions  just  "showed  themselves"  or  came 
to  them  in  sudden  bursts  of  insight  Indeed,  many  famous 
scientists  describe  grasping  a  solution  instantaneously  and  as 
a  whole,  and  then  having  to  face  the  arduous  task  of  putting  the 
idea  already  completely  conceived  into  an  appropriate  verbal 
form  to  share  with  others.  Although  this  section  does  not 
pretend  to  present  an  exhaustive  and  comprehensive  account 
of  visualization  by  scientists  and  inventors,  the  few  examples 
that  follow  make  a  convincing  point  about  the  value  of  visual- 
ization in  cognition. 

Albert  Einstein's  unique  methods  of  wrestling  with  the 
most  puzzling  problems  of  physics,  such  as  light  taking  on 
characteristics  of  both  particles  and  waves  simultaneously,  are 
among  the  most  well-known.  Einstein  was  known  for  using 
"thought  experiments"  to  work  out  problems  in  a  uniquely 
nonverbal  manner.  Perhaps  his  most  famous  thought  experi- 
ment was  imagining  what  it  would  be  like  to  ride  on  a  beam  of 
light.  This  allowed  him  to  make  the  conceptual  leap  of 
"seeing"  light  as  though  it  were  in  static  form.  This  helped  him 
to  resolve  the  paradoxes  underlying  what  was  to  become  his 
Special  Theory  of  Relativity.  In  another  example,  he  imagined 
how  two  people  would  describe  the  behavior  of  a  light  flashing 
inside  a  moving  truck  if  one  person  was  riding  in  the  truck  and 
the  other  was  standing  on  the  street  to  help  understand  the 
absolute  nature  of  the  speed  of  light 

The  German  chemist  August  Kekull  is  another  scientist 
famous  for  his  reports  of  imagery.  He  often  described  how 
atoms  appeared  to  "dance  before  his  eyes."  He  is  said  to  have 
discovered  the  ring-like  molecular  structure  of  benzene  by 
gazing  into  a  fire  and  seeing  in  the  flames  a  ring  of  atoms 
looking  like  a  snake  eating  its  own  tail.  His  accounts  of 
problem-solving  through  dreamlike  visual  imagery  are  echoed 
in  the  case  stories  of  many  other  scientists,  including  Isaac 
Newton. 

Roger  Shepard  (1988)  offers  one  of  the  most  interesting 
and  detailed  discussions  of  how  famous  scientists  and  inven- 
tors have  been  predisposed  to  visualization  in  their  acts  of 
creative  imagination  and  discovery.  Beyond  those  of  Einstein 
and  Kekutt,  Shepard  describes  the  creative  inventiveness  of 
dozens  of  famous  scientists,  such  as:  Michael  Faraday's 


1019 


8 

1 

6 

3 

5 

7 

4 

9 

2 

Figure  J.  A  "magic  square''  Aii  number  across,  down,  and  diagonal 
sum  to  15. 

visualization  of  the  lines  of  magnetism:  Nikola  Tesla's  inven- 
tion of  the  self-starting  induction  motor;  Omar  Snyder's  solu- 
tion to  the  containment  problem  of  uranium  in  the  Manhattan 
project;  James  Watson's  conception  of  the  double-helix  shape 
of  DNA;  and  Richard  Feynman's  invention  of  "Feynman's 
diagrams'*  for  use  in  quantum  electrodynamics.  A  curious 
similarity  of  many  of  these  famous  thinkers  is  that  they  were 
often  able  to  grasp  their  solutions  instantly  as  a  whole. 

Based  on  his  investigations,  Shepard  described  a  "com- 
posite caricature'*  of  individuals  who  have  reported  extraordi- 
nary instances  of  visual-spatial  creative  imagery.  Three  com- 
monalties can  be  found  in  these  people's  early  formative  years. 
Many  were  kept  home  from  school  in  their  first  years  and  had 
limited  contact  with  peers  of  their  own  age.  Many  were  below 
average  in  verbal  ability,  such  as  language  development. 
Finally,  most  were  fond  of  engaging  in  play  with  concrete 
physical  objects,  such  as  blocks,  cubes,  and  mechanical  mod- 
els. Most  of  these  skills,  abilities,  and  strategies  were  devel- 
oped apart  from  the  educational  systems  of  their  day. 

Shepard  goes  on  to  suggest  some  provocative  implica- 
tions of  this  composite  profile  of  a  highly  creative,  nonverbal 
thinker.  Working  in  private  without  much  contact  with  formal 
educational  institutions  (such  as  schools),  these  people  are 
likely  to  engage  in  unorthodox  and  nontraditional  thinking 
which,  unfortunately,  may  be  met  with  disapproval  or  punish- 
ment in  a  traditional  classroom.  These  individuals  are  more 
1  ikely  to  engage  in  concrete  visual  imagery,  instead  of  the  more 
abstract,  verbal  strategies  commonly  promoted  in  the  schools. 
Consequently,  these  people  are  likely  to  bring  the  unique 
human  competency  of  spatial  intuition  and  manipulation  to 
bear  on  a  problem.  Finally,  the  dominance  of  visual  imagery 
in  problem-solving  is  more  likely  to  trigger  the  motivational 
and  affective  forces  thought  to  be  more  aligned  with  visual 
elements  of  the  human  psyche. 

Shepard  also  discusses  some  of  the  educational  implica- 
tions of  his  research  on  the  creative  imagery  of  famous  scien- 
tists. Not  surprisingly,  he  criticizes  traditional  education  for 
failing  to  promote  visually-based  creative  tendencies  in  chil- 
dren. As  children,  the  scientists  he  studied  equated  learning 
with  becoming  "engrossed  in  a  direct,  interactive  exploration 
of  such  objects  and  events..."  and  were  "unconstrained  by 
conventional,  verbalized,  and  rigidly  compartmentalized  in- 
terpretations..." (p.  181).  He  suggests  education  must  find  a 
way  to  nurture  creative  imagination  without  sacrificing  formal 
education,  though  the  two  often  appear  to  be  in  direct  conflict 
with  one  another. 


Other  Examples 

This  section  briefly  considers  the  historical  value  of  visu- 
alization as  an  aid  to  problem-solving  from  very  diverse  areas 
of  inquiry.  These  examples  are  presente.,  in  chronological 
order  beginning  with  the  Cholera  epidemic  in  the  mid- 1 800*s 
and  ending  with  the  new  science  of  Chaos,  an  emerging  field 
of  study  in  which  people  and  computers  work  together  in 
partnership  through  scientific  visualization. 

The  CholeraEpidemic  of  London  in  the  mid-  l80Ols.  One 
of  the  classic  examples  of  how  visualization  aided  human 
problem-solving  was  Dr.  John  Snow's  plotting  of  cholera 
deaths  in  the  mid- 1 800* s  on  a  map  of  London.  The  obvious 
clustering  of  deaths  around  (he  Broadstreet  water  pump,  as 
shown  in  Figure  2,  sufficiently  convinced  authorities  to  re- 
move the  pump's  handle  even  though  a  direct  link  had  not  been 
made  between  the  disease  and  a  contaminated  water  supply. 
Within  days,  the  epidemic  in  the  London  neighborhood  ended 
(Tufte.  1983). 

Wilbur  Wright's  Wing-Warping  System.  The  story  of  the 
invention  of  the  airplane  contains  many  interesting  insights  to 
design  and  technology.  One  simple  example  aptly  illustrates 
the  role  of  everyday  visualization  and  imagination.  Control- 
ling an  airplane  in  its  three-dimensional  environment  (i.e. 
pitch,  roll,  and  yaw)  was  oneof  the  most  difficult  problems  the 
Wright  Brothers  and  others  faced.  The  Wrights  had  already 
successfully  used  a  rudder  to  control  yaw  and  an  elevator  to 
control  pitch,  but  were  having  much  difficulty  controlling  the 
plane's  yaw  (i.e.  the  motion  along  the  axis  going  through  the 
fuselage  from  the  plane's  nose  to  tail).  Wilbur  Wright  solved 
this  problem  of  human-controlled  powered  flight  by  seeing  the 
remarkable  -wing- warping"  system  used  by  the  Wright  Broth- 
ers' Flyer  I  at  Kitty  Hawk  while  holding  and  twisting  an  inner 
tube  box  in  his  bicycle  shop  in  Dayton,  Ohio.  Moo  1  man  ( 1980) 
writes: 

Then  one  day  in  the  latter  part  of  July  1 899.  while  Wilbur 
was  alone  in  the  bicycle  shop,  a  customer  came  into  buy  a 
new  inner  tube.  Wilbur  chatted  with  the  customer  awhile, 
idly  toying  with  the  empty  inner  tube  box  before  throwing 
it  away;  as  hetalked  he  realized  that  he  had  absently  twisted 
the  ends  of  the  narrow  cardbc  ard  box  in  opposite  directions. 
When  the  customer  left,  Wilbur  tore  off  the  ends  of  the  box 
and  saw  in  his  mind's  eye  a  pair  of  biplane  wings,  vertically 
ngid  yet  twisted  into  opposing  angles  at  their  tips  (p.  1 12). 

Plate  Tectonics.  It  was  long  believed  that  the  earth's 
continents  had  remained  in  their  general  positions  immedi- 
ately after  the  surface  of  the  earth  cooled,  even  though  the 
vertical  shape  of  the  land  masses  had  been  subjected  to  dra- 
matic changes,  as  evidenced  by  finding  dried-up  sea  beds  in  the 
mountains.  However,  some  peculiar  facts  puzzled  scientists. 
For  example,  identical  fossil  evidence  was  found  on  the  east 
coast  of  South  America  and  the  west  cost  of  Africa.  These  were 
explained  in  various  ways  such  as  migratory  animals  who  used 
now  submerged  land  bridges.  Other  evidence,  such  as  striking 
geological  similarities,  were  much  more  difficult  to  explain 
away. 

In  19 IS.  Alfred  Wegener,  a  German  meteorologist,  pro- 
posed a  different  solution  (Burke*  1985),  He  noticed  how 
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Figure  2.  The  famous  doi  map  of  Dr.  John  Snow  plotting  the 
cholera  deaths  in  London  in  relation  to  neighborhood  water  pumps. 
This  map  provided  strong  evidence  that  the  water  in  the  Broad 
Street  pump  was  contaminated. 


many  of  the  outlines  of  the  continents  seem  to  fit  together  like 
a  giant  jigsaw  puzzle,  the  most  dramatic  example  being  how 
the  east  coast  of  South  America  seemed  to  fit  the  west  coast  of 
Africa.  Perhaps,  he  suggested,  at  one  time  there  was  one  large 
land  mass  which  eventually  broke  apart.  At  the  time. 
Wegener's  proposal  was  ridiculed  by  geologists.  The  idea  of 
comments  drifting  throughsolid  rock  seemed  ludicrous.  It  was 
not  until  about  50  years  later  that  Wegener's  visual  solution 
was  accepted  based  on  the  discovery  of  mid- ocean  ridges  and 
evidence  of  sea  floor  spreading.  Rock  samples  increase  in  age 
proportionally  to  the  distance  in  which  they  are  taken  away 
from  the  mid-ocean  ridges.  Also,  evidence  indicates  that  the 
earth's  magnetic  field  apparently  changes  direction  every  few 
hundred  million  years  or  so.  Since  rocks  maintain  their 
magnetic  "finger  print"  it  is  possible  to  correlate  the  ages  of 
rock  with  their  inherent  magnetic  direction.  When  sections  of 
the  sea  floor  arc  mapped  using  this  magnetic  evidence,  mag- 
netic "stripes"  appear  on  each  side  of  mid-ocean  ridge  showing 
that  sections  of  the  sea  floor  alternate  in  their  magnetic  direc- 
tion. As  molten  rock  emerged  from  the  mid-ocean  ridges  and 
cooled,  it  captured  the  earth's  magnetic  direction  at  that  geo- 
logical period  of  time.  Current  theories  now  accept  that  the 
earth's  crust  is  made  up  of  distinct  plates  that  "float"  on  the 
earth's  mantle.  Wegener  elegant  solution  was  based  on  the 
most  visual  of  available  evidence. 

Armor  Plating  of  World  War  11 A  ircraft.  A  very  practical 
example  of  visualisation's  role  in  problem-solving  comes 
ti  World  War  II.  A  novel  strategy  was  used  to  better  armor 
.ibat  planes.  The  bullet  holes  on  returning  aircraft  were 
plotted  on  crude  pictures  of  the  planes.  Using  this  information. 


it  was  determined  to  add  armor  to  planes  in  places  other  than 
those  indicated  by  the  bullet  holes.  The  idea  was  that  since  it 
was  assumed  that  the  planes  were  all  hit  more  or  less  at  random, 
the  planes  that  did  not  return  must  have  been  hit  in  vital  places 
not  marked  on  the  picture  (Wainer,  1992). 

The  Science  of  Chaos.  Some  consider  computers  as  the 
tool  by  which  the  world  will  be  turned  into  a  mechanized  and 
inhuman  place  to  live,  but  a  contrasting  view  considers  the 
computer  as  our  liberator  by  performing  the  tedious,  routine 
tasks  poorly  suited  to  humans  and  freeing  us  to  more  fully 
realize  our  potential.  This  collaboration  between  people  and 
computers  is  perhaps  best  illustrated  in  the  founding  of  the  new 
science  of  Chaos,  which  is  the  study  of  nonlinear  systems 
(Gleick.  1987).  Such  systems,  though  seemingly  random  and 
haphazard  on  the  surface,  actually  have  a  hidden  order  lurking 
below.  The  universe  is  inundated  with  such  systems,  though 
some  of  the  best  examples  are  from  the  everyday  world, 
including  the  weather,  flags  waving  in  the  breeze,  ribbons  of 
smoke,  and  dripping  water  faucets.  Even  human  problem- 
solving  is  believed  to  be  a  nonlinear  system.  The  study  of 
nonlinear  systems  has  only  been  made  more  accessible  with 
the  advent  of  computers.  The  patterns  of  complex,  nonlinear 
systems  often  only  show  themselves  when  the  raw  data  is 
converted  into  visual  form.  The  innate  human  ability  of  pattern 
recognition  in  combination  with  the  computer's  forte  of  work- 
ing through  millions  of  iterations  with  complex  data  structures 
have  allowed  many  of  the  mysteries  of  chaotic  systems  to  be 
explored  and  better  understood. 

One  of  the  most  interesting  examples  is  fractal  geometry, 
where  a  pattern  repeats  itself  to  infinity,  such  as  the  figure 
known  as  the  "Sierpinski  gasket"  shown  in  Figure  3.  This 
figure  is  created  through  an  astonishingly  simple  set  of  rules 
(Michael  Bamsley.  as  cited  in  Gleick,  1987.  referred  to  this  as 
'The  Chaos  Game").  The  game  starts  by  drawing  three  "game 
dots"  on  a  piece  of  paper  (such  as  those  at  the  three  comers  of 
an  equilateral  triangle).  Another  dot.  call  this  the  starting  cor. 
is  drawn  at  random  on  the  piece  of  paper.  Then,  randomly 
choose  one  of  the  three  game  dots  (such  as  by  throwing  a  die). 
Next,  carefully  draw  another  dot  at  die  midpoint  between  (he 
randomly  chosen  game  dot  and  the  starting  dot.  This  midpoint 
now  becomes  the  next  starting  dot.  Finally,  repeat  this  proce- 
dure for  thousands  of  trials.  Although  the  rules  of  the  Chaos 
game  might  lead  one  to  expect  a  random  collection  of  dots  on 
tlw  paper,  a  "hidden  order"  emerges  when  the  numeric  infor- 
mation is  converted  into  a  visual  form — the  Sierpinski  gasket. 

Of  course,  few  people  are  willing  to  invest  the  time  or 
energy  necessary  to  the  play  the  Chaos  game  (and  its  many 
variations).  That  is  where  the  computer  comes  in.  Computers 
are  wonderfully  equipped  to  handle  the  tremendous  number  of 
accurate  calculations  necessary  in  the  chaos  game  whereas 
people  are  wonderfully  equipped  to  interpret  the  visual  pat- 
terns that  emerge.  The  science  of  chaos  represents  a  powerful 
partnership  between  people  and  machines  by  letting  each  do 
what  they  do  best. 

Visualization  Gone  Awry:  Some  Counter 
Examples 

It  seems  only  fair  to  consider  some  examples  of  where  the 
powersof  visualization  actually  worked  against  some  people. 
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Figure  3.  The  Sierpinski  Gasket:  This  fractal  geometric  figure 
repeats  itself  to  infinity. 


Visualization,  like  perception,  is  not  like  a  camera  objectively 
capturing  images  on  film.  Interpretation  and  understanding  are 
continually  filtered  through  one's  entire  knowledge,  values, 
and  bel  iefs.  People  often  see  and  imagine  what  they  wantxo  see 
and  imagine.  Visualization,  like  any  cognitive  process,  is 
greatly  influenced  by  prior  knowledge.  Two  examples  are 
presented  here  where  visualization  led  to  erroneous  conclu- 
sions. The  result  of  one  example  led,  however  briefly,  to  wide 
pandemonium,  excitement,  and  fantastic  stretches  of  the 
imagination  —  Percival  Lowell's  report  of  seeing  artificially 
constructed  canals  on  Mars.  The  repercussions  of  the  other 
example,  in  contrast,  dramatically  changed  the  world  forever 
—  Columbus'  voyage  to  find  a  westward  route  to  China  and 
India. 

Percival  Lowdl  was  a  prominent  astronomer  at  the  turn  or 
the  century  (he  founded  che  Lowell  Observatory  in  Arizona). 
Lowell  became  interested  in  the  planet  Mars  based  on  early 
observations  in  1877  by  Giovanni  Schiaparelli  mat  showed 
some  interesting  fine  lines  on  the  Martian  surface  (Ronan, 
1983).  Lowell  subsequently  studied  the  planet  in  the  early 
1900  s  using  the  most  sophisticated  telescopic  equipment 
available  at  that  time.  Lowell,  like  Schiaparelli,  also  observed 
the  peculiar  long  crossing  lines  on  the  Martian  landscape. 
Lowell  became  convinced  that  these  were  the  remnants  of 
canals  constructed  by  some  ancient  civilization.  The  purpose 
of  the  canals,  Lowell  inferred,  was  a  desperate  attempt  to  bring 
water  down  from  the  polar  caps  to  the  desert-like  continental 
areas.  Unfortunately,  the  "canals"  turned  out  to  be  an  optical 
illusion.  This  example  is  a  classic  case  of  jumping  to  conclu- 
sions based  on  initial  and  ambiguous  evidence,  known  by 
cognitive  psychologists  as  "top-down  processing1'  —  initial 
information  triggers  an  early  interpretation  against  which  all 
subsequent  information  is  judged.  This  is  an  important  psy- 
chological mechanism  that  helps  us  to  find  order  and 
organization  in  an  otherwise  chaotic  environment  Of  course, 
sometimes  it  works  against  us.  This  same  phenomena  pro- 
duces people's  tendency  to  see  dead  presidents  in  fluffy  white 
clouds. 

The  story  of  Columbus  is  not  as  amusing  or  innocent,  if 
only  because  his  adventures  changed  forever  the  global  view 
of  the  world.  The  intent  here  is  not  to  discuss  the  details  of  his 
trip  or  its  ramifications,  but  simply  why  Columbus  chose  to 


make  it  in  the  first  place.  It  seem  *  Jwt  the  nost  compelling 
reason  Columbus  dared  to  risk  svjch  an  expedition  i«  simply 
that  he  greatly  underestimated  us  s.-c  of  the  earth  coupled 
with  a  dramatic  misconception  of  what  r^oporilon  of  the  earth 
consisted  of  water  and  knd  {of  course,  we  sh  ^uld  not  forget 
how  important  the  potential  wealth  and  fane  figured  in  his 
decision-making  as  well)  (Dor-Nt.,  199iy.  Had  he  accepted  an 
accurate  account  of  these  two  facts,  it  is  almost  certain  he  and 
his  sponsors  would  have  felt  the  trip  impractical  and  foolhardy 
ar  best  and  impossible  at  worst. 

Historians  belie  'e  that  Columbus'  views  were  heavily 
influenced  by  the  writings  of  Marco  Polo,  Pierre  d' Aiily,  Pope 
Pius  II,  Pliny,  and  Ptolemy.  Both  Polo  and  d'Ailly  overesti- 
mated the  size  of  Asia  considerably.  The  question  of  the 
earth '  s  circumference  had  been  a  source  of  scientific  debate  for 
centuries,  going  back  at  least  to  the  Greeks.  The  true  figure  is 
60  nautical  miles  per  degree  of  longitude  at  the  equator. 
Though  Eratosthenes  had  come  close  to  estimating  the  true 
circumference  of  the  earth  (about  j'9.5  nautical  miles  per 
degree),  Columbus  chose  figures  closer  to  that  estimated  by 
Ptolemy  (SO  nautical  miles).  Columbus  also  and  inexplicably 
downsized  the  figure  even  further,  to  about  45  nautical  miles 
per  degree  of  longitude.  Therefore,  Columbus  envisioned  a 
globe  that  was  only  about  two  thirds  its  true  size  and  most  of 
that  he  thought  was  covered  by  land.  Columbus  estimated  a 
journey  from  the  Canary  Islands  to  Japan  to  be  only  about 
2,400miles  instead  of  the  11, 000 miles  it  actually  is.  UsingtmV 
information,  Columbus  successfully  argued  his  case  for  such 
a  journey,  The  result  of  his  journey  was,  of  course,  the 
accidental  discovery  of  a  new  continent  though  he  died  believ- 
ing instead  that  he  had  reached  islands  near  the  coast  of  Asia 
Of  course,  one  could  argue  that  Columbus  used  these  miscon- 
ceptions on  purpose  to  persuade  King  Ferdinand  and  Queen 
Isabella  of  Spain  to  fund  the  trip  as  well  as  to  Find  a  crew  even 
partly  wil  ling  to  join  him.  For  example,  Columbus  admitted  to 
falsifying  information  kept  in  the  log  to  alleviate  the  crew's 
fears  (Fuson,  1987).  Even  if  this  were  to  be  true,  Columbus' 
use  of  visualization  for  deception  deserves  equal  attention. 

Conclusions  and  Implications 

The  purpose  of  this  paper  has  been  to  present  some  simple 
examples  of  how  visualization  has  served  as  an  impou.jit 
problem-solving  tool  for  people  throughout  history.  An  his- 
torical context  not  only  provides  the  most  dramatic  examples 
of  visualization  in  problem-solving,  but  also  helps  to  promote 
reflection  on  one  of  our  most  distinctly  human  abilities. 
Though  we  may  never  adequately  understand  the  psychology 
of  visualization,  it  will  and  should  continue  to  serve  as  one  of 
our  most  versatile  problem-solving  tools.  Instructional  de- 
signers, teachers,  and  all  educators  are  therefore  encouraged  to 
consider  innovative  visualization  strategies  to  nurture  the 
creative  problem-solving  process.  Concrete,  visual  solutions 
should  not  be  considered  inferior  to  those  that  are  abstract.  Of 
coutse,  the  twocounter  example*  also  serve  to  caution  against 
unwarranted  and  inappropriate  applications. 

Despite  the  relatively  small  number  of  examples  pre* 
sented  here,  one  $oor<  discovers  the  pervasive  nature  of  visual- 
ization in  scientific  discovery  and  invention.  The  examples 
presented  hetc  were  meant  only  to  suggest  the  case  for  the 
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continued  value  of  visualization  strategics  and  should  not  be 
mistaken  for  an  exhaustive  survey.  The  list  of  examples  not 
accounted  for  is,  of  course,  large.  Some  domains,  like  geom- 
etry, are  inherently  spatial  in  nature  and  have  their  own 
visualization  histories  to  tell.  Some  other  notable  examples 
missing  from  this  paper  include  the  following:  Kepler's 
formulation  of  the  laws  of  planetary  motion;  the  discovery  of 
chemical  "fingerprints"  of  elements  as  lines  in  a  spectrum 
(another  good  example  of  pattern  recognition);  and  the  spatial 
arrangement  of  the  periodic  table  of  elements.  In  contrast, 
some  concepts  seem  impossible  to  visualize,  such  as  the  idea 
of  curved  space  or  a  physical  universe  consisting  of  more  than 
three  dimensions,  concepts  suggested  by  Einstein  and  modern 
day  physicists.  Similarly,  other  historical  problems,  such  as 
accurately  describing  the  motion  of  a  projective  through  space 
(such  as  cannon  balls),  provide  interesting  insights  to  people's 
attempts  to  visualize  phenomena  that  have  few  visual  clues. 

There  are  many  important  implications  that  one  can  draw 
from  this  review.  The  trends  in  multimedia  learning  environ- 
ments, especially  those  that  are  computer-based,  are  slowly 
moving  from  verbal  to  visual,  from  analog  to  digital,  and  from 
passive  to  interactive.  The  implications  for  t^e  learning 
process  are  more  far  reaching  than  media  dominance,  how- 
ever, especially  when  the  computer's  processing  and  graphical 
abilities  are  considered.  The  computer  has  the  potential  to 
become  one  of  our  most  important  cognitive  tools,  similar  to 
the  way  paper  and  pencil  reduced  demands  on  human  memory. 
Highly  visual  computer-based  learning  environments,  such  as 
Geometer's  Sketchpad  and  Interactive  Physics*  allow  indi- 
viduals to  grapple  with  sophisticated  ideas  from  math  and 
science  in  visual  ways  that  are  allat  once  concrete  and  intuitive. 
Computers  and  people  working  closely  as  '"partners  in  cogni- 
tion'* have  the  potential  for  fundamental  qualitative  changes  to 
how  we  view  human  cognition  (Salomon,  Perkins.  & 
Globerson.  1991). 

The  implications  for  instructional  designers  are  likewise 
exciting  and  challenging.  Highly  visual  and  interactive  com- 
puter-based tools  may  allow  the  user  to  take  on  an  unprec- 
edented role  in  the  design  process.  Rather  than  merely  strive 
for  learner-centered  instruction  that  takes  into  account  indi- 
vidual differences,  instructional  technology  may  be  poised  to 
let  the  user  become  a  true  "co-designer"  of  our  learning 
environments.  Similarly,  some  of  the  emerging  technologies, 
such  as  virtual  reality,  point  to  design  considerations  that  have 
never  been  asked  before  (Heim.  1993).  Some  of  these  are  also 
among  the  most  exciting,  though  we  should  be  quite  cautious 
early  on.  The  nature  of  how  people  construct  their  own  reality 
may  become  muddled  when  immersed  in  \  isually  overwhelm- 
ing environments.  Thcqucstionof  whether  "telepresence.0  the 
state  of  interacting  in  one  location  (even  an  imaginary  one) 
while  physically  located  in  another,  will  be  an  acceptable  state 
of  "existence"  in  the  future  should  not  be  casually  considered. 
Some  feel  that  telephones  already  achieve  a  degree  of 
telepresence  since  people  focus  on  their  conversation  with  the 
person  on  the  other  end  and  not  on  the  distance  which  separates 
them.  Yet.  when  brought  to  the  awareness  level  of  an  adult, 
there  is  no  mistaking  one's  physical  reality  while  using  phone 
technology.  This  distinction  may  become  blurred  with  the 
advent  of  virtual  reality  technology.  This  is  a  particularly 
important  issue  as  our  children  begin  to  experience  virtual 


reality.  There  is  the  risk  that  their  cognitive  development  of 
reality  may  become  contused,  similar  to  very  young  children 
who  become  angry  that  "grandma  cannot  come  through  the 
phone  receiver"  to  be  with  them  at  that  very  moment  Intellec- 
tual development  of  space  and  time  are  important  issues  to 
consider.  The  implications  of  these  technologies  demand 
attention  and  guidance  by  instrucuonal  technologists  today  in 
preparation  for  tomorrow. 
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Table  1.  Information  Location  and  Use  Activities  (Find,  Engage,  Extract)  by  Children  in  Print  and 
Multimedia  Treatment  Groups 
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Table  2.  Information  Location  and  Use  Activities  (Find,  Engage,  Extract)  by  Children  and  Adult 
Multimedia  Treatment  Groups 
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Group 

n 

Eng 

[age 

Text 

Nontext 

Tot             %  Mean 

Tot              %  Mean 

Pr/Ch 

43 

758             55%  17.6 

i  s 
617            45%      |  14.4 

Mm/Ch 

46 

978            25%  20.8 

2873           75%  62.5»M 

•  p<.05 
p<.01 
»•*  p<.001 
••••  p<.0001 


Table  3.  Engaging  Activities  for  Text  and  Nontext  Information  by  Children  in  Print  and 
Multimedia  Treatment  Groups 
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Nontext 
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Tot 

% 

Mean 
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% 

Mean 

Mm/Ch 

46 
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25% 

20.8 

2873 

75% 

62.5 

Mm/  Ad 

35  . 

1521 

28% 

435**** 

3998 

72%  ■ 

114.2"* 

♦  p<.05 
p<.01 

***  p<.001 
p<.0001 


Table  4*  Engaging  Activities  for  Text  and  Context  Information  by  Children  and  Adult  Treatment 
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p<.05 
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Table  5.  Total  Motivation  Scores,  Value  and  Expectancy  Subscale  Post-PR  Mean,  and  Pre-Post 
Change  Scores  for  Children  in  Print  and  Multimedia  Treatment  Groups 
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p<.01 
p<.0Ol 
♦♦♦♦  p<.0001 


Table  6.  Total  Motivation  Scores,  Value  Expectancy  Subscale  Post-Pre  Mean,  and  Pre-Post 
Change  Scores  for  Children  and  Adult  Multimedia  Treatment  Groups 
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